17 KARLSRUHER BEITRÄGE ZUR REGELUNGS- 
UND STEUERUNGSTECHNIK 


Lukas Kölsch 


Dynamic Incentives 
for Optimal Control of 
Competitive Power Systems 


NIT zueishing 


Lukas Kölsch 


Dynamic Incentives for Optimal 
Control of Competitive Power Systems 


Karlsruher Beiträge zur 
Regelungs- und Steuerungstechnik 
Karlsruher Institut für Technologie 


Band 17 


Dynamic Incentives for Optimal 
Control of Competitive Power Systems 


by 
Lukas Kölsch 


SICHT Fusishine 


Karlsruher Institut für Technologie 
Institut für Regelungs- und Steuerungssysteme 


Dynamic Incentives for Optimal Control of Competitive Power Systems 


Zur Erlangung des akademischen Grades eines Doktor-Ingenieurs 
von der KIT-Fakultät für Elektrotechnik und Informationstechnik des 
Karlsruher Instituts für Technologie (KIT) genehmigte Dissertation 


von Lukas Kölsch, M.Sc. 


Tag der mündlichen Prüfung: 10. Dezember 2021 
Hauptreferent: Prof. Dr.-Ing. Sören Hohmann 
Korreferent: Prof. Dr.-Ing. Andreas Rehkopf 


Impressum 
NCI Scientific 
Publishing 
Karlsruher Institut für Technologie (KIT) 
KIT Scientific Publishing 


Straße am Forum 2 
D-76131 Karlsruhe 


KIT Scientific Publishing is a registered trademark 
of Karlsruhe Institute of Technology. 
Reprint using the book cover is not allowed. 


www.ksp.kit.edu 


© OO This document - excluding parts marked otherwise, the cover, pictures and graphs — 
OAE is licensed under a Creative Commons Attribution-Share Alike 4.0 International License 
(CC BY-SA 4.0): https://creativecommons.org/licenses/by-sa/4.0/deed.en 


© OO) The cover page is licensed under a Creative Commons 
Ra Attribution-No Derivatives 4.0 International License (CC BY-ND 4.0): 
https://creativecommons.org/licenses/by-nd/4.0/deed.en 


Print on Demand 2022 - 
Gedruckt auf 100 % Recyclingpapier mit dem Gütesiegel „Der Blaue Engel” 


ISSN 2511-6312 
ISBN 978-3-7315-1209-7 
DOI 10.5445/KSP/1000147941 


Preface 


This dissertation has been written during my work as research assistant at the Institute of 
Control Systems (IRS) at Karlsruhe Institute of Technology (KIT). 


First and foremost, I would like to express my deepest gratitude to my doctoral supervisor 
Prof. Dr.-Ing. Sören Hohmann for giving me the opportunity of pursuing my PhD under 
his chair, his excellent scientific guidance throughout my time at the IRS and his permanent 
openness to explore and find new research fields. I also owe sincere gratitude to Prof. Dr.-Ing. 
Andreas Rehkopf for acting as a second examiner and for his personal encouragement during 
the final phase of my work. 


I am highly indepted to my former master’s students (and now colleagues) Felix Strehle, 
Pol Jané Soneira, and Lukas Rausche, whose exciting scientific acumen in general and careful 
proofreading of this dissertation in particular have made a crucial impact on shaping the 
manuscript in its present form. Likewise, I would like to thank Dr.-Ing. Stefan Krebs and 
Dr.-Ing. Martin Pfeifer for their ongoing interest and motivation, which always pushed me to 
find the right path. I am very thankful for the best office colleagues one could imagine, namely 
Florian Köpf, Simon Rothfuß, and Manuel Schwartz, with whom I was allowed to experience 
many prosperous discussions of both scientific and non-scientific nature, discovering the 
spontaneous self-therapeutic effect of collaborative whiteboard discussions. I would also like 
to thank Rishabh Vyas, who helped me numerous times with special linguistic questions about 
the final manuscript even at the most pathological times of the night, as well as all my other 
undergraduate and master’s students, whose willingness to write their own thesis under my 
supervision was instrumental in generating numerous new ideas. 


Furthermore, I am very grateful for the financial support provided by the German Research 
Foundation as part of the DFG Priority Programme “Hybrid and Multimodal Energy Sys- 
tems”. 


Finally, my heartfelt and very special thanks go to my parents Viola and Roland for their 
unconditional mental support during the writing process and throughout my whole life. 


Karlsruhe, December 2021 Lukas Kölsch 


Abstract 


Incentive mechanisms can play a pivotal role in accelerating technologically challenging 
transformation processes such as the energy transition. The aim of those incentives is to 
provide an environment where the interplay of each competitor’s individually optimal actions 
is also globally optimal in the sense of an overall objective. The dissertation at hand presents 
a feedback control framework towards optimal incentive mechanisms for today’s and future 
power systems in the triad of system stability, economic efficiency, and grid support. A key 
innovation of the developed control scheme is the incorporation of temporally and spatially 
differentiated real-time price signals that result from the solution of static and dynamic 
optimization problems. Inclusion of locally available measurement information, consistent 
co-modeling of the underlying physical network including resistive losses, and continuous- 
time formulation of all subsystems pave the way towards a real closed-loop incentive control 
framework. Particular importance is attached to a rigorous separation between market and 
network participants, respecting the principle of unbundling. After comprehensive analysis 
of the resulting closed-loop system, the control scheme is further adapted to a novel real- 
time congestion management system. Two case studies illustrate practical advantages of 
the developed approach compared to existing concepts. The port-based system modeling, 
the absence of any centralized control authority, and the ability for automatic, decentralized 
regulation of all prices across the entire network finally enable seamless extensibility by 
additional incentive components. 


Kurzfassung 


Technologisch herausfordernde Transformationsprozesse wie die Energiewende können durch 
passende Anreizsysteme entscheidend beschleunigt werden. Ziel solcher Anreize ist es hierbei, 
ein Umfeld idealerweise so zu schaffen, dass das Zusammenspiel aller aus Sicht der beteiligten 
Wettbewerber individuell optimalen Einzelhandlungen auch global optimal im Sinne eines 
übergeordneten Großziels ist. Die vorliegende Dissertation schafft einen regelungstechnis- 
chen Zugang zur Frage optimaler Anreizsysteme für heutige und zukünftige Stromnetze im 
Zieldreieck aus Systemstabilität, ökonomischer Effizienz und Netzdienlichkeit. Entscheidende 
Neuheit des entwickelten Ansatzes ist die Einführung zeitlich wie örtlich differenzierter 
Echtzeit-Preissignale, die sich aus der Lösung statischer und dynamischer Optimierungsprob- 
leme ergeben. Der Miteinbezug lokal verfügbarer Messinformationen, die konsequente Mit- 
modellierung des unterlagerten physikalischen Netzes inklusive resistiver Verluste und die 
durchgängig zeitkontinuierliche Formulierung aller Teilsysteme ebnen den Weg von einer 
reinen Anreiz-Steuerung hin zu einer echten Anreiz-Regelung. Besonderes Augenmerk der 
Arbeit liegt in einer durch das allgemeine Unbundling-Gebot bedingten rigorosen Trennung 
zwischen Markt- und Netzakteuren. Nach umfangreicher Analyse des hierbei entstehenden 
geschlossenen Regelkreises erfolgt die beispielhafte Anwendung der Regelungsarchitektur 
für den Aufbau eines neuartigen Echtzeit-Engpassmanagementsystems. Weitere praktische 
Vorteile des entwickelten Ansatzes im Vergleich zu bestehenden Konzepten werden anhand 
zweier Fallstudien deutlich. Die port-basierte Systemmodellierung, der Verzicht auf zen- 
tralisierte Regeleingriffe und nicht zuletzt die Möglichkeit zur automatischen, dezentralen 
Selbstregulation aller Preise über das Gesamtnetz hinweg stellen schließlich die problemlose 
Erweiterbarkeit um zusätzliche optionale Anreizkomponenten sicher. 
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Rei stator resistance 

S vector of inter-cell complex power flows 

S control weight matrix 

Si complex power injection 

Si complex power flow 


Sm apparent power flow over line m 

te end time 

t time 

Tio, i d-axis open-circuit transient time constant 
Tao, i d-axis open-circuit subtransient time constant 
Toi q-axis open circuit transient time constant 
Teo,i q-axis open-circuit subtransient time constant 
u input vector 

UAC,i AC-side voltage magnitude of inverter 

UDC,i DC-side voltage of inverter 

Uti magnitude of excitation voltage 

Uzi terminal (internal) voltage magnitude 

U; nodal voltage phasor 

U; nodal (external) voltage magnitude 

U! transient internal voltage magnitude 

Ur subtransient internal voltage magnitude 
Geom nominal voltage 

Gee voltage setpoint 

g^ maximum deviation of voltage magnitude 

v eigenvector 


V(x) (control-)Lyapunov function 


Greek Symbols 


XIX 


Symbol Description 

V(x, w) extended control-Lyapunov function 
V*(a) value function 

w vector of weights for extended control-Lyapunov function 
x state vector (of closed-loop system) 
Xai d-axis synchronous reactance 

X a. i d-axis transient reactance 

Xii d-axis subtransient reactance 

Xij specific series reactance 

Ani reactance of the mains connection 
Lp plant state vector 

Kai q-axis synchronous reactance 

X 4i q-axis transient reactance 

Xii q-axis subtransient reactance 

XTi transformer reactance 

Y nodal admittance matrix 

y output variable 

Yii parallel admittance of power line 
Ysh,i shunt admittance 

z co-state 

Zu series resistance of power line 


Greek Symbols 


Symbol Description 

a learning rate 

F; moment of inertia of swing equation 

Vi rotor angle in static reference frame 

ĝi absolute rotor angle in synchronous rotating reference frame 
ö(t) Dirac delta function 

€“ restoration signal 

Ç controller co-state 

Nij multiplier for neighboring nodal prices 

(S) magnetic flux 

0; bus voltage phase angle 

Ve i rotor angle in synchronous rotating reference frame 

Oni pseudo rotor angle in synchronous rotating reference frame 
Vij bus voltage angle difference 

L communality factor 

R diagonal matrix of participation factors 

Kk participation factor for cell k 


XX Calligraphic Symbols 
Symbol Description 
Ki, participation factor for cell k without correction 
KL participation factor for cell k without regularization and correction 
Ar wholesale price in cell k 
A Lagrange multipliers for equality constraints 
A; price at node 7 
Ke uniform price 
AR retail price 
H Lagrange multipliers for inequality constraints 
V vector of virtual power flows for frequency control 
Va vector of virtual power flows for balance of payments containment 
= vector of basis functions 
€ controller state 
T circle number 
w scaling factor for cell-specific cost function 
Pi active loss of node i 
0; reactive loss of node i 
o balance of payments error 
T diagonal matrix of time constants 
Y Lagrange multiplier of modified optimal control 
® overall active transmission loss 
©; overall active transmission loss in cell k 
bk generalized effort for zonal pricing 
Pi load angle 
x scaling factor for reactive power sharing 
y auxiliary variable for synchronous machine limits 
p generalized flow vector for zonal pricing 
Wi deviation of nodal frequency from nominal value 
Omi angular velocity of mechanical rotor 
Wm i deviation of angular velocity of mechanical rotor 
Qn i nominal value of angular velocity 


Calligraphic Symbols 


Symbol Description 

A Laplacian matrix 

B(x) open ball around & 

Cim congestion factor of line m 

E« set of communication edges 

Ê set of inter-cell communication edges 

Eck set of communication edges of cell coordinator k 


Script Symbols XXI 
Symbol Description 
Ep set of plant edges 
Ê set of inter-cell edges 
E set of boundary edges 
Car set of non-boundary edges 
I inverter-type 
Fe objective function for adaptation 
L load-type 
M neighborhood 
N; neighbors of i in G (undirected) 
Nbi neighbors of i in 9, (undirected) 
Q quadric 
R nonlinear resistive structure 
S synchronous machine-type 
T tangent cone 
U input space 
Ve set of synchronous machine- and inverter-type nodes 
Vek set of synchronous machine- and inverter-type nodes in cell k 
Vr set of inverter-type nodes 
Ve set of synchronous machine- and load-type nodes 
Ve set of load-type nodes 
Vp set of nodes of plant system 
Vs set of synchronous machine-type nodes 
Vw set of wholesale prosumer nodes 
Ywr set of nodes of wholesale prosumers 
Vz set of nodes in cell k 
W set of wholesale prosumers 
A state space 
Z set of cells 


Script Symbols 


Symbol 


Description 


Coni 


V 2 R E R 


overall communication graph 

aggregated cell communication graph 
communication graph of cell coordinator k 
graph of physical system 

aggregated cell graph 

Hamiltonian (Pontryagin function) 
imaginary part 

objective function 


XXII Typewriter Symbols 


Symbol Description 

L Lagrangian 

£ state-dependent integrand of objective function 
R real part 


Doublestruck Symbols 


Symbol Description 

A; set of actions of player i 

L solution set 

N set of natural numbers 

P set of players 

R set of real numbers 

wr set of weights leading to positive control-Lyapunov function candidate 

Yr set of states and weights leading to positive control-Lyapunov function 
candidate 

Yayı cylindric set 

YS maximum cylindric set 


Typewriter Symbols 


Symbol Description 

c overall cost function 

Cgi cost for power generation at node i 

Ces convexified cost function for power generation at node 7 
cf cost of retail prosumer in cell k 

Cr cost of wholesale prosumer 7 

J objective function 

L Lagrangian of Lagrange-type performance index 

Pei profit at node i 

Pk profit of cell coordinator k 


P- profit of wholesale prosumer 7 

Q quadratic function 

Rr revenue of wholesale prosumer 7 
W welfare function 


Other Symbols 


XXIII 


Other Symbols 


Symbol 


Description 


Cx. oS B SCO 


all-zeros matrix 
all-zeros vector 

empty set 

all-ones vector 
imaginary unit 
Hadamard product 
Cartesian product 
penalty/barrier function 


Indices and Exponents 


Index 


Description 


complex conjugate 
optimizer 
pseudoinverse 
convexified 
cellular 

time derivative 
complex variable 
upper limit 

lower limit 
communication 
d-axis 

generation 

node (instance) 
node-to-node (instance) 
injected 

node (instance) 
cell (instance) 
cell-to-cell (instance) 
consumption 

cell (instance) 
mechanical 

edge (instance) 
nominal 

physical system 
q-axis 


XXIV 


Operators 


Index Description 
R retail 
S stator 
sh shunt 
w wholesale 
be power plant operator (instance) 
Operators 
Operator Description 
V gradient 
= not 
arg argument 
bd boundary 
cl closure 
coll column vector 
diag diagonal matrix 
im image 
int interior 
ker kernel 
proj canonical projection 
II gradient projection 


| lee) 


absolute value 
Euclidean norm 
maximum norm 


1 Introduction 


The harnessing of electrical energy is among the most groundbreaking achievements in human 
history. Its permanent and widespread local availability by means of an interconnected power 
system is a cornerstone of our modern society and has become an indispensable condition 
for any civilizational, ie. cultural, technological, and economic progress. As a result, this 
civilizational progress itself has consequences for the energy supply system: Stemming from 
a growing awareness of the negative impact of fossil energy production on the environment, 
and provoked by the climate debate of recent years, the idea of sustainability has gained an 
increasing presence. Accordingly, our natural environment has faced the repercussions of 
technological progress, thus in particular the emissions of greenhouse gases as well as the 
depletion of natural resources, should be given greater focus [RLP* 15]. This understanding 
has led to a number of worldwide political declarations of intent, according to which the 
ecological footprint is desired to be drastically reduced. As part of the European Green Deal, 
in 2021 the European Climate Law [Cou21] was passed, according to which the EU member 
states are obliged to reduce their net greenhouse gas emissions to zero by 2050. Moreover, it 
enforces a reduction in greenhouse gas emissions of minus 55%' by the year 2030. 


The power generation sector holds a prominent role in climate protection, as it not only 
accounts for the largest share of global greenhouse emissions to date [BBM* 14, p. 516; 
MEH" 17, p. 7], but also has the greatest quantitative potential for substitution compared with 
other sectors [DKH19, p. 3]. As a consequence, many countries have set ambitious goals for 
the next decades to shift away from conventional fossil-fueled power generation towards 
renewable and low-carbon energy sources. Germany’s reduction targets are consolidated in 
the Climate Protection Plan 2050. For the power generation sector, this directive imposes a 
drastic reduction from 280 to 175 million tonnes of CO, equivalents between 2020 and 2030°. 
These actions are intended to increase the share of renewables in gross electricity consumption 
in Germany? from 32% in 2015 to 65% in 2030 [Deu14, KM17, Deu20]. 


From a systems viewpoint, these ambitious policy roadmaps are accompanied by significant 
techno-economic disruptions, namely, the increasing displacement of conventional, fossil- 
fueled power plants in favor of power generation from renewable energy sources (RESs), 
such as wind, photovoltaic, biomass or run-of-river plants, leading to a structural change 
in the feed-in characteristcs: Since the majority of RES power plants are based on energy 
sources whose availability is weather-dependent [SP18], no exact forecasts can be made about 
the expected feed-in. This growing uncertainty manifests itself in an increasing volatility of 
the residual load, meaning that the difference between the current demand and the current 
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related to the reference year 1990. 

Sector-specific targets on an annual basis are imposed in a federal law, see [Deu19, Annex 2]. 

Likewise, there are national measures and legislative acts for stimulating green electricity in the U.S. [EMGĦ 21], 
China [HZZ20], Canada [Gov16], India [Int20], and Scandinavia [Nor17]. An overview of the measures in the 
individual EU member states is given in [Eur20]. 
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generation by RES power plants fluctuates to an increasingly large extent. However, it is 
important to note that this increasing volatility has not only a temporal, but also a geographical 
dimension. Unlike conventional power plants, which can be located deliberately at load 
centers [Sch14, p. 11], renewable generation units are usually located wherever geographic 
conditions allow an economically viable harvest of e.g. wind or solar supply. This creates the 
undesirable phenomenon of locally non-integrable power during periods of high solar exposure 
or high wind intensity [MGMH14]. Likewise, load centers with geographic conditions that 
are expected to be less profitable for power plant operators may suffer from ongoing deficits 
in local net generation. This consequently will result in significant and increasing amounts of 
compensation flows via the transmission lines [ZPM13]. 


In light of this upcoming scenario of increasing temporal and local volatility, there is serious 
reason among decision-makers as well as in the scientific community to expect that the 
existing power transmission and distribution system, in its current form, will not be able to 
keep pace with these developments. Furthermore, it is evident that the existing technical and 
regulatory system infrastructure is noticeably becoming stressed. Until now, this fact has 
already manifested itself e.g. in increasingly volatile spot market prices [RN20], increased 
incidence of line congestion leading to critical grid interventions [Sta19, p. 42], reverse 
power flows [NCB* 18, HRR20], and increasingly high amounts of net imports and exports of 
electricity over long distances [KO20, p. 164]. 


The most evident technical means of counteracting these issues is through the installation of 
local energy storages (see [DBK* 18, Gür18, BMC* 20] for comprehensive reviews of prospec- 
tive technologies and their economic viability), accompanied by a massive grid expansion. 
However, apart from the long time horizons from planning to launch’, oversizing transmission 
lines according to a worst-case calculation is known to not be economically viable, espe- 
cially if this worst-case scenario can only be roughly estimated. Apart from these technical 
challenges, there is also a strong need for redesign of the existing regulatory and market 
framework to accommodate the high intermittency of RESs while ensuring overall system 
stability [KO20, p. 164f.; Sch21, p. 6f.]. 


Moreover, increased competition among the flexible participants is a useful mechanism to 
stimulate the participants’ own strive for innovation. At the same time, excesses in consumer 
prices have to be prevented in order to avoid undermining people’s confidence in the viability 
of renewable energy generation and thus missing the targets of the whole energy transition. 
Consequently, a proper mechanism design should deliver real-time signals to all competitors 
that incentivize supply during times of particularly high demand and in geographic regions 
where it is of utmost necessity from a “global” network perspective. Although the current 
structure of the energy market, dominated by normative interventions such as feed-in tariffs 
or balancing group (BG) contracts, is not capable of providing such investment signals that 
internalize network effects, first studies also show that a mere adoption of locally differentiated 
prices may lead to false incentives as well, unless the slowly reacting settlement times of 
existing electricity markets are overcome (see e.g. [Con19]). 


4 Due to extensive tendering and approval processes, the average time required for a transmission line construction 


permit is 7 years, with an upper quartile of 14 years [BL11, p. 4]. Likewise, electrochemical storages with fast 
reaction times such as battery energy storage systems will presumably not be available on a large scale within 
the next 20 years [FSK 20, p. 2]. 
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At this point, scientific contributions from the area of automatic control theory can contribute 
to “close the loop” between market and network (i.e., between the pricing mechanism and 
the physical interaction of participants across the power grid) and to rigorously analyze 
the interdependencies that arise in this process. The dissertation at hand is thus devoted 
to developing a holistic feedback control framework for large-scale interconnected power 
systems with competitive participants which ensures stability of the overall system. By 
applying a market-based architecture, the controller is intended to pursue economically efficient 
operating points without any need for additional supervision by a higher-level authority. 
At the same time, the control framework should be able to support the grid by taking the 
geographical component of momentary infeed into account. Thereby, continuous valuation 
of each participant’s individual contribution to a stable network operation should provide 
instantaneous price incentives for local oversupply or undersupply. Despite this conceptual 
redesign towards real-time signaling, the proposed framework is intended to be a genuine 
brownfield approach, i.e., it should neither tighten the existing action space of competitive 
network participants nor require a redesign of physical infrastructure. Hence, the overall 
objective of this dissertation can be summarized as follows: 


Objective. Design of a unifying, market-based feedback control strategy to enforce a 


stable, economically efficient, and grid-supportive operation of future power systems. 


The outline of this dissertation is as follows (see Fig. 1.1). Chapter 2 reviews the current 
state of the art, formulates the existing research gaps, and outlines the specific contributions 
of the dissertation aligning with the above objective. Subsequently, these specific contribu- 
tions are elaborated in Chapters 3-5. In Chapter 6, effectiveness and performance of the 
presented overall control framework is evaluated by means of case studies on a realistic IEEE 
57-bus benchmark system. Finally, a conclusive discussion on the contributions is given in 
Chapter 7. 
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Figure 1.1: Outline of this dissertation. 


2 Related Work and Research Contributions 


This chapter forms the basis for the scientific contributions of the subsequent chapters by 
providing a comprehensive survey on recent advances in competitive power systems. Section 
2.1 analyzes the status quo of today’s electricity markets and outlines recent tendencies. 
Section 2.2 surveys the state of research. Section 2.3 summarizes the main findings of the 
previous sections by means of three specific research gaps. Finally, Section 2.4 presents the 
main contributions of this dissertation that result from these research gaps. 


2.1 Today’s Liberalized Electricity Markets 


From the beginning of electricity production until the 1980s, the power grid was characterized 
by vertically integrated electricity companies, which concentrated all aspects of power supply, 
from generation and distribution to grid operation, under the same roof. However, as power 
systems became increasingly interconnected in the 1900s, this concept proved to be insufficient 
to take the increasing importance of power flows over long distances into consideration. For 
instance, a power plant operator (PPO) that simultaneously acts as a local system operator 
would always retain both the incentive and the opportunity to prevent competitors from 
entering the market. While it is economically desirable to create competition between PPOs, 
the system operators (i.e. owners of the transmission and distribution infrastructure), by 
contrast, constitute natural monopolists [GS17, p. 264f.] due to their high fixed costs and low 
variable costs’. From an economic perspective, the most effective way to handle natural 
monopolies is through regulation. For this purpose, the ownership of some part of the network 
is auctioned off by the state on the basis of long-term concessions, and a regulatory authority 
constantly monitors compliance with these contracts. This separation of different roles has 
led to the need of unbundling, which includes the following key aspects [BGM 14, p. 7]: 


a) Legal Unbundling: Corporate splitting of former integrated electricity companies into 
system operators, and generation and supply companies. 


b) Operational Unbundling: Authority and operational decision-making of the generation 
and supply companies must not be influenceable by system operators. 


c) Informational Unbundling: Information from each company must either be disclosed on 
a non-discriminatory basis to all other participants or treated confidentially. Information 
received from other participants must be treated confidentially as well. 


Practically speaking, it is not useful to construct multiple power line infrastructures from different companies in 
parallel. 
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Figure 2.1: Status quo of key stakeholders in Germany’s power system. The arrows mark the typical flow direction of 
physical power (solid lines), information (dashed lines), or capital (triangle lines). Color-coded components 
mark those effects that arise by increased infeed of RESs and are discussed in Subsection 2.1.2. 


Vv 
v 
v 
11414141004144400044040L1LIL I ALALAAAI AL AI AIT 


As a consequence, there exist individual companies with clearly defined roles and narrow 
information boundaries. In this process, power generation and supply continue to be left to 
free competition, while system operators are regulated by the state authority. 


2.1.1 Overview and Participants 


Fig. 2.1 shows a schematic overview of the relevant stakeholders in Germany’s power system® 
as it exists today. The power network infrastructure can be divided into the transmission 
network (typically with nominal voltages above 100 kV) and the distribution network (nominal 
voltage below or equal to 100 kV). While the transmission network operates as a supraregional 
transport and interconnection layer, the distribution network is conceived for subordinate, 


é Although nomenclature is based on the usual terminology in Germany, a fairly similar structure can also be 


found in other European countries as an outcome of EU-wide harmonization. 
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Table 2.1: Electricity markets in Germany. 


Type Product Timeframe Volume Tick 
WEM EEX Futures >lw 1MW 
WEM EPEX SPOT Day-Ahead 1h 0.1 MW 
WEM EPEX SPOT Intraday 15 min 0.1 MW 
BM Minute Reserve 15 min 1 MW 
BM Secondary Control Reserve 5min 1MW 
BM Primary Control Reserve 30s 1MW 


regional distribution of electrical energy to the point of consumption. Consequently, trans- 
mission grids tend to be more meshed than distribution grids, which are typically operated as 
open ring or radial grids, and are characterized by an R/X ratio close to zero. 


Large-scale conventional power plants (CPPs) are directly connected to the transmission grid 
via step-up transformers. The aim of each CPP is to deliver a certain amount of electrical 
power within a specific period of time. There are various opportunities for selling the produced 
power: On one hand, the own supply can be offered on the Wholesale Electricity Market 
(WEM). The WEM is an energy-only market, which means that only the energy actually to 
be supplied at a certain time is offered. On the other hand, there exists a balancing market 
(BM), where each PPO’s production capacity can alternatively be offered to system operators 
as a means for stabilizing interventions, e.g. after disturbances or faults. The BM is a capacity 
market, which means that it trades the option of having a certain amount of electrical power 
delivered for a certain period of time. In fact, the offered power packages are standardized in 
terms of spatial and temporal quantization, and depending on the specific time scale, both 
WEM and BM divide into specific sub-markets. Table 2.1 provides an overview of the existing 
electricity markets in Germany. In a fully unbundled power system, CPPs are owned by 
independent companies which are purely profit-driven. Whether marketing its own electrical 
power is more profitable on the WEM or the BM depends on the current market forecast of 
the individual CPP. 


The demand side can be divided into wholesale consumers, which cover their own demand 
directly via the WEM, and retail consumers, which pay a fixed electricity price quoted by an 
electricity supplier (ES). The measurement of the actually consumed power is done over a 
5-minute interval for wholesale consumers and at an annual interval for retail consumers. 
Between measurement intervals, consumption of the latter group is estimated based on 
standard load profiles (SLPs). The ES has to purchase respective amounts of power on the 
WEM in the exact amount corresponding to the (estimated or actual) consumption. 


Each supply or demand point of the power system is assigned to a BG. The balance responsible 
party (BRP) has to provide a zero balance between supply and demand for each quarter-hour 
interval. This can be achieved optionally by means of the own assets, or with the help of 
purchases or sales on the WEM. 
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The transmission system operator (TSO) is the owner of a certain part of the transmission 
network infrastructure’ and is responsible for its maintenance and expansion. Furthermore, 
the TSO is responsible for stability of the own network. For this purpose, it uses its own 
measurements in the network as well as information from the subordinate distribution system 
operators (DSOs)®. By aggregating the transmitted schedules of the BRPs, the TSO performs an 
after-market power flow calculation for the upcoming day and, if necessary, has the authority 
to intervene in the schedules of the power plants in the event of forecast line overloads 
(redispatch) or to prevent non-integrable generation by RESs on times of high net generation 
surpluses (feed-in management). Furthermore, the TSO shall ensure that any current imbalance 
is remedied by requesting balancing power from the BM or by applying its own stabilizing 
measures, such as staging of transformers or installation of compensator elements, e.g. static 
VAR compensators or flexible AC transmission systems (FACTSs). 


2.1.2 Recent Developments 


The increased penetration of RESs into the transmission or distribution grid (see red compo- 
nents in Fig. 2.1) has a significant implication on the existing market. The main developments 
are reported below. 


Fixed Remuneration of RESs Infeed 


To encourage a more rapid investment in RESs, numerous financial incentive instruments have 
been created. For instance, RES operators are either paid a fixed (i.e. guaranteed) remuneration 
or an additional premium per amount of power fed in (see orange arrows in Fig. 2.1). The 
power fed into the grid is marketed by the TSOs. However, since revenue from sales on the 
WEM is typically far less than the guaranteed remuneration for RESs [CHMG14, p. 304], the 
resulting extra costs must be compensated by retail consumers and wholesale consumers? 
paying an additional consumer levy to the local DSO. Likewise, if feed-in management is 
required due to an oversupply by renewables, the respective RESs are compensated for the 
lost profit, which is also covered by the consumer levy. This levy is taking an ever-increasing 
share of the net electricity price. In Germany, it averaged 20.4% for retail consumers in 2021 
and is considered a significant driver of current and future electricity prices [Bun21, p. 19.]. 
This is a paradoxical fact, since the actual variable costs of electricity generation by RESs are 
significantly lower than those from fossil energy sources. 


In Germany, there are 4 TSOs. All of them are part of the European Supergrid of the European Network of 
Transmission System Operators for Electricity (ENTSO-E), consisting of 42 TSOs from 35 countries in total. 

8 Within the ENTSO-E network there are more than 2300 DSOs, 880 of which are located in Germany [TM15, p. 4; 
PFAT 19, p. 11]. 

If wholesale consumers manage to be classified as “energy-intensive”, they can exempt themselves from the 
consumer levy [Deu14]. 
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Redispatch and Copper Plate Assumption 


The remuneration policy for RESs discussed above has another side effect that arises particu- 
larly in the case of weakly developed grids or long transmission paths: If CPPs are displaced 
from the merit order during periods of high solar and/or wind generation, there is a local 
undersupply of load centers. While these may be compensated arithmetically by the respective 
RESs, significant spatial disparities between the locations of generation and consumption 
may occur, e.g. when offshore wind turbines ramp up, thereby superseding CPPs far from the 
coast. If these disparities exceed the physical transmission capacity limits of the network, re- 
dispatch has to be ordered by the responsible TSO (see blue arrow in Fig. 2.1). This redispatch 
mechanism is cost-based, i.e. both the down-regulated and up-regulated power generators 
are each compensated pairwise for their lost profits or additional costs to be incurred. In fact, 
central Europe has seen a surge in the occurence of redispatch events over the last decade 
[Fek21, p. 5f.; EE21], and this trend is assumed to continue [Sta19, p. 42]. 


Cost-based redispatch provides numerous inappropriate incentives for PPOs from a macroeco- 
nomic and environmental perspective: For an individual PPO, the compensation mechanism 
makes the specific location of generation and thus the role of a certain power plant to the 
overall network completely irrelevant!®. Instead, the PPOs’ investment decisions are only 
governed by minimizing their own fixed costs, which often results in a preference for rural 
areas far from load centers. At the same time, PPOs have no incentive to replace existing, 
low-efficiency power plants at critical load centers with higher-efficiency power plants, since 
cost reimbursement by the TSO is always guaranteed, even if the bid falls well outside the merit 
order. Furthermore, studies have shown that PPOs can anticipate the occurence of a redispatch 
event with very good accuracy solely based on seasonal patterns and local weather measure- 
ments, even without any knowledge of the prevailing network conditions [Sta19, Chapter 5]. 
This creates several speculative trading opportunities for PPOs that are unfavorable from 
a global systems view: For instance, if it can be anticipated that a specific power plant will 
be curtailed by the TSO, it is advantageous for this PPO to submit a lower bid in order to 
be scheduled, but then to be reimbursed for the curtailment without having to physically 
deliver the amount of power. Conversely, if a certain power plant is likely to be requested 
after-market by the TSO for redispatch, strategic PPOs will bid too high to raise their own 
expected compensation payments. Further speculative trading strategies following from good 
predictability are discussed in [Sta19, p. 53]. 


Balancing Energy 


The increasing volatility of power supply increases the difficulty for BRPs to keep their BGs 
balanced. If a BRP delivers less power than scheduled, the TSO is obliged to procure the 
difference on the BM and later charges its own costs as a balancing energy payment (see violet 
arrows in Fig. 2.1). Likewise, if the BRP delivers more power than scheduled, it receives a 
balancing energy payment in the same amount by the TSO. This mechanism is thus another 
speculative trading opportunity: In times of high spot market prices it is advantageous for the 


10 This phenomenon is called copper plate assumption, which refers to the fact that from the PPO’s point of view, 


the whole power network is reduced to a single copper plate with unlimited transmission capacity. 
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BRP to sell its own power on the WEM instead of keeping its own BG balanced. Analogously, 
in times of low spot market prices, it is preferable to purchase power on the WEM and earn 
the difference to the received balancing energy payment as arbitrage profit [MRF11]. This 
exploitation pattern counteracts the efforts of the TSO to keep the network balanced and is 
likely to endanger the stability of the network. In recent years, several critical incidents in the 
ENTSO-E network have been reported which trace back to strategic behavior of the BRPs 
(cf. [JW15, KH19, PLd20)). 


2.1.3 Discussion 


The above examples show that under the impact of increasing RESs infeed, the existing 
electricity market in its current form is tending towards undesirable interactions between 
stakeholders. Thereby, the (normative) role ofthe TSO on electricity markets is increasing, 
such that the original requirement for unbundling is only partially fulfilled. In particular, 
while legal unbundling is still given, there are an increasing number of situations where 
the TSO has to intervene in PPOs’ decision making or where the receivable information 
empowers PPOs to exploit their knowledge to the disadvantage of other participants, thus 
violating both operational and informational unbundling. These trends result in a more 
frequent occurrence of critical network situations and thus in an increasing instability ofthe 
network. The economic inefficiencies also create substantial extra costs for consumers (see 
Fig. 2.1). In addition, cost-based redispatch and copper plate assumption act as key drivers for 
PPOs to act contrary to what would be necessary and desirable from a network perspective. 


In order to resolve the shortfalls shown in Subsection 2.1.2, a certain tendency towards even 
stronger normative intervention by the regulatory authority can be observed [EE19]. However, 
such interventions do not adequately address the intrinsic problem and often reach their 
target far too late". 


Moreover, numerous position papers suggest a shortening of the current time frames (see 
Table 2.1) towards a real-time interaction of the stakeholders. In a real-time scenario, the 
actions (i.e. outputs) of individual stakeholders are based on continuous measurements of the 
own state and environment. Thus the overall interaction of all participants on the physical 
network an via different markets can be interpreted as a feedback control system comprising a 
large number of internal control loops with different response speeds. However, there is also 
a wide consensus across the research community that retaining the existing WEM structure is 
not sufficient to prevent the tendencies described in Section 2.1.2. For instance, the occurence 
of redispatch is always an indicator of structural market failure [Ris21, p. 58], as it may trigger 
network participants at critical points to further aggravate the (anticipated) congestion. 


Consequently, the following section provides a comprehensive overview over existing control- 
theoretic approaches which are able to make vital contributions towards a stable, economically 
efficient, and grid-supportive electricity market design. 


11 For example, after a particularly serious case of balancing energy speculation in Germany in June 2019, high 


penalties were imposed on the BRPs involved by the responsible Federal Network Agency [Bun20], albeit the 
majority of cases are likely to remain undetected [LC12, p. 259f.]. 
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Figure 2.2: Overview of main phenomena in power systems [Saul1, MBB12, OW15, CLI*+17]. The focus of this 
dissertation is on timescales between 10 ms and 1 d and is highlighted in gray. 


2.2 State of Research 


Fig. 2.2 provides an overview of the main phenomena in electrical power systems ranging 
from 10~° s to 10° s. Typically, the common term power system stability is associated 
with timescales smaller than 15 min. Stability is said to be given, “if, after being subjected 
to a disturbance, all state variables [of the network] recover to (possibly new) steady-state 
values which satisfy operational constraints [...] without the occurence of involuntary load 
shedding” [FCSP* 20]. According to the latest IEEE-PES definition, power system stability 
encompasses the sub-tasks frequency stability, voltage stability, rotor angle stability, converter- 
driven stability, and resonance stability [HMR* 20]. Among these sub-tasks, frequency and 
voltage stability play the most important role for system operators”. By contrast, measures 
for economically efficient and grid-supportive operation take place on timescales above 1s (see 
Fig. 2.2). They range from supply-side optimization (economic dispatch) and demand-side 
management to day-ahead scheduling and long-term grid planning. 


In the remainder of this section, the main research branches devoted to one or more of the 
three sub-objectives from Chapter 1 are presented. First of all, Subsection 2.2.1 provides an 
overview of the main control paradigms and system architectures that are being discussed 
in the power systems literature. Focusing on slower timescales, Subsection 2.2.2 outlines 
the main ideas and results for the economic operation on supply- and demand-side. Finally, 
Subsection 2.2.3 reviews the latest research on congestion management. Subsequently, an 
in-depth review on the state of research on faster timescales is given for frequency control 
(Subsection 2.2.4), voltage control (Subsection 2.2.5), and the combination of both (Subsection 
2.2.6). 


12 In particular, rotor angle and converter-driven stability are typically maintained by the owner of the respective 


facility by means of inner control loops on a faster time scale [Tv16, WBLt 14, RYI* 16]. Accordingly, both 
categories are not considered for the remainder of this dissertation. Likewise, resonance stability encompasses 
subsynchronous phenomena being addressed by constructional measures (cf. [NFM* 11, LS15, VG16]), which is 
beyond the scope as well. 
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2.2.1 Overview of Control Architectures 


Historically, the power system followed a strict top-down approach not only in its physical 
structure but also concerning the responsibility of system operators. In particular, the TSO 
alone was responsible for the procurement of stability, while the DSO only undertook subordi- 
nate control tasks such as transformer staging and power line switching. However, triggered 
by the replacement of CPPs by RESs, the distribution network has evolved from a weakly 
equipped supply grid towards an interactive and self-sufficient smart grid, encompassing 
decentralized communication via broadband connections, bidirectional prosumers with local 
storages, sector coupling, smart meter instrumentation for retail customers, and advanced 
cyber-security protection [SRRD15, DEAT 17, KD18, ABAH* 20]. 


The current literature suggests different approaches for transitioning the former top-down 
control hierarchy between TSO and DSO towards a more agile and autonomous mode of 
operation. The concept most widely discussed is the microgrid (MG) topology'*. Originally 
presented in [Las02], a single MG denotes a specific section of the network’, which is 
connected to the superordinate grid by a single point of common coupling. In normal (“islanded”) 
mode, each MG is isolated from the main network and thus both supply and stabilization 
are provided decentrally by means of local storage and generation capacity only. If islanded 
operation is no longer possible, e.g. in case of severe outages or load shifts, the MG switches 
to grid-connected mode, thus compensating the imbalance via the main grid. Through the 
principle of subsidiarity, each MG can be controlled separately without having to account for 
interdependencies with neighboring MGs. This factor has contributed to an increasing number 
of real-world studies where the MG concept is implemented!°. A current overview of in-field 
experiences is given in [MMR 21, p. 4]. Major challenge in MG operation, however, is the 
re-synchronization with the superordinate grid, which is highly nontrivial and yet in addition 
often occurs especially at that moment when the MG is in a severe state [DRVA10, ZS10]. 
Moreover, self-sufficient operation in islanded mode necessitates that generation and storage 
capacity be generously sized in order to always meet its own demand. It thus has to be 
critically acclaimed that the physical separation into isolated portions comes at the price of 
considerably higher investment costs to guarantee worst-case redundancy separately for each 
MG--in fact, computational overhead is simply shifted to resource overhead. 


From the perspective of market participants, the increased availability of communication 
infrastructure has enabled the development of virtual power plants (VPPs) [Dv03, MR11, 
NY21], virtually aggregating geographically diverse small-scale distributed energy resources 
(DERs) to one single asset. This allows the VPP operator to internally reduce volatility and 


13 All of these building blocks are either in process of field testing, or a nation-wide rollout in some countries is 


upcoming in the next few years. See the given references for practical evidence. 

To date, no formal definition for microgrid exists within the research community. The one presented here is 
adopted from [OMSE+ 14] and [SFO15, p. 52ff.]. However, there exist alternative definitions such as [TS12], 
which do not require a point of common coupling, but only require the MG to act as a single controllable entity 
with respect to the superordinate grid. 

Typically on the scale of a medium voltage network. Similar structures at lower voltage levels, e.g. within 
households, are defined as nanogrid by a majority of authors. 

In fact, even at the beginning of the electricity age in the late 19th century, all urban power grids were operated 
in islanded mode. Even today, this is still in practice in many remote areas e.g. in Canada or Russia. In this 
respect, it would be more appropriate to speak of a revival of the MG. 
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unpredictability through the effect of diversification, and thus be accessed similarly to large- 
scale CPPs for the purpose of stability procurement. In contrast to the MG concept, the overall 
network remains physically interconnected. Several authors distinguish between the two 
different roles of Technical VPPs (TVPPs) and Commercial VPPs (CVPPs). In this notion, TVPPs 
pool the regional generation capacity of small-scale DERs within a certain area. They have 
detailed measurement information about the state of the local network and use their own 
resources to provide system services to TSOs and DSOs. Conversely, CVPPs are aggregators 
of DERs from possibly distant regions. They have no information about the network and place 
bids on the WEM based on their aggregated generation profile. The conceptual separation 
into CVPPs and TVPPs has the advantage that small- and micro-scale DERs gain access for 
both the WEM and for procurement of ancillary services on the BM. However, the previous 
structure of TSOs, DSOs, and BRPs which was criticized in Section 2.1 (cf. Fig. 2.1) typically 
remains unchanged. An alternative approach to organize a fully decentralized power system 
without the need for large redundancies is the web-of-cells (WoC) concept [MBR* 17, LHK19]"”. 
Here, the overall network is partitioned into cells operated by autonomous cell coordinators 
(CCs), each exclusively responsible for stability within its own cell. The WoC concept is able 
to reduce complexity by means of information barriers between CCs, albeit without cutting 
the physical interconnections between cells. Furthermore, there exists no superordinate 
instance anymore, thus CCs communicate in a peer-to-peer manner. Although this approach 
is interesting, most studies have only focused on the conceptual formalism itself. Yet to date, 
the literature on specific control schemes tailored to the WoC scenario is scarce (see e.g. the 
contributions [RCT17, MRHD17]). 


2.2.2 Real-Time Economic Dispatch and Demand-Side Management 


A major implication of the copper plate assumption is that the existing WEM does not take 
into account the physical and operational conditions (e.g. power flow equations and line flow 
limits), but relies on the “fiction of a congestion-free market area“ [HSMT19, p. 13]. Therefore, 
real-time economic dispatch approaches aim to replace the existing WEM with a (feedforward 
or feedback) controller, incorporating these physical and operational conditions in an optimal 
power flow (OPF) problem!®, which is solved at specific points in time. 


In the remainder of this subsection, we distinguish between centralized approaches, which 
pursue a single overall objective, approaches from noncooperative game theory’, where each 
network participant is modeled as a player, with no contracts possible between them, and 
approaches from cooperative game theory, which involve binding contracts between players. 
A comprehensive review of the recent state of research in economic dispatch and demand- 
side management is provided in the survey paper [WYL21]. For a detailed discussion of 
continuous-time methods, see [DBSPG19]. 


17 A number of sources refer this concept as grid of MGs or networked MGs, cf. [WL20]. 


18 OPF is a well-studied problem class in both theory and application. Early OPF algorithms [DT68, AS74, HG91] 
are based on Newton-type methods, while today’s solvers usually rely on active set and interior point methods 
[NW06, Chapter 14; Tay15, Chapter 3; FR16, p. 1185] or on convex relaxation techniques of the OPF problem, 
such as linear relaxation (DC-OPF) [TCL12] or semidefinite relaxation [Low14]. 

An overview of basic concepts and main terminology of game theory used throughout this dissertation is 
summarized in Section A.2 in the appendix. 
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Centralized and Model-Predictive Approaches 


Aggregating all objectives and constraints into a single, centralized OPF problem allows 
to circumvent the issues from Subsection 2.1.2, since no after-market interventions such 
as redispatch or consumer grid tariffs are necessary. Model predictive control (MPC) is an 
advanced control technique originally stemming from the process industry. It involves solving 
a repeated numerical solution of a snapshot of the optimization problem with a preferably fast 
sampling time, and then applying this numerical solution to the controller output within the 
current time interval upon repeating the calculation in the next time step. As the steady-state 
mappings derived from the dynamic constitutive equations of the physical system typically 
correspond to the AC power flow equations, the control architecture of MPC emerges as a 
repeated offline solution of static AC-OPF problems whose feasible set can be referred to as 
the nonconvex power flow manifold (see e.g. [BD15]). The ability to also incorporate predictive 
data like weather or load forecasts in the OPF problem makes MPC particularly suitable for 
optimal dispatch of power grids with high penetration of RESs [HMMD13, p. 5827f.]. Extensive 
research on the application of MPC for optimal dispatch of power networks is conducted for 
low voltage distribution networks [KPP+17, PRK+17, DSE* 21], industrial consumers and 
households [SGAM18, SFC* 18, FS21] and islanded power systems [PRG14, OCN* 17, SSCt 19]. 
While the authors in [PRG14] use computer simulations to show the effectiveness of the 
approach, power hardware-in-the-loop demonstrators are used in [PRK*17] and even field- 
site demonstrations are available in [FS21], which highlight the overall practicality of MPC 
for real-world optimal dispatch. 


In spite of their widespread presence in the context of power networks, it must be critically 
acknowledged that the accuracy and thus feasibility of the obtained solutions is strongly 
dependent on the accuracy of the precomputed system model. Conversely, as the complexity 
ofthe underlying system model increases, so does the computational complexity [BBPS09]. 
This, in turn, prevents the choice of reasonably fast sampling times. 


In response to these practical imperfections, which are very common in real-world scenarios 
(cf. [PZS*18]), stochastic MPC methods have gained a foothold in power system research over 
the last 20 years, employing the representation of model uncertainties as stochastic constraints. 
However, the robustness gained by including all possible model deviations and disturbances 
comes at the price of fairly “conservative” solutions that do not fully explore the entire feasible 
set and thus often yield profits which are significantly below the maximum achievable profits, 
whereby, the computational complexity for finding a solution increases strongly once again 
as compared to the deterministic framework. Beyond academic examples, the usability of 
stochastic feedforward models for real-time optimization of interconnected power systems 
with intermittent and volatile feed-in characteristics thus seems questionable. 


Moreover, in both classical and stochastic MPC, a centralized, omniscient system operator 
is assumed to have both full knowledge about all system states and access to all actuators. 
This assumption is unfavorable for large-scale power systems both in terms of computational 
complexity and vulnerability, thus limiting the applicability of classical MPC to small-scale 
power systems with a moderate number of parameters. 


2.2 State of Research 15 


Recent developments in distributed model predictive control (DMPC) methods [VHRW08, 
CSML13] allow to efficiently decompose the optimal dispatch problem into subproblems, such 
that local agents with geographically limited measurement and control access optimize only a 
part ofthe system. [SESJ13] proposes aDMPC algorithm for economic dispatch using Dantzig- 
Wolfe decomposition. [XXM19] presents a cost-minimal DMPC scheme for grid-connected 
low voltage networks, the performance of which is validated on a power hardware-in-the-loop 
platform. In [RCPP20], a dual decomposition method is applied for DMPC-based optimal 
scheduling of interconnected migrogrids. These approaches suffer from the fact that the 
distributed agents aim to jointly minimize a global objective function. However, in fully 
unbundled power systems, network participants usually are not committed to acommon goal 
and do not take cooperative decisions. Instead, they operate selfishly by maximizing their 
own profits, i.e., try to minimize their own objective function. Distributed continuous-time 
controllers for radial power systems are presented in [ZP15]. In [ZLP15], the controllers are 
extended to meshed topologies. However, both approaches again rely upon the assumption of 
zero transmission losses. 


In general, a proper accounting for transmission losses is a key challenge of all OPF-based 
methods, especially at distribution grid level, due to inherent nonconvexity of lossy AC- 
OPF problems and their resulting non-existence of appropriate Lyapunov candidate functions 
[SALB05; Sch15, Remark 6.2.2]. In [MNRQ17], losses are approximated by robust loss coefficients 
in order to provide a distributed solution to the economic dispatch problem. Improved loss 
coefficients formulas, which also consider reactive power flows, are presented in [HYC* 18]. 


Noncooperative Game-Theoretic Approaches 


Noncooperative game theory is concerned with selfishly driven network participants and 
provides tools and methods on how to maximize their own profit while taking into account 
the other players’ strategies. The main solution concept in noncooperative game theory is the 
Nash equilibrium (cf. Definition A.6), which is used to characterize a state in which no player 
can achieve a higher profit by unilaterally changing its strategy. 


Former research focused on noncooperative approaches with highly simplified models of 
network participants, which allows Nash equilibrium seeking to be resolved using linear 
programming techniques, see e.g. [SLL02, WY13] for generator dispatch or [ZHB12] for 
demand-side management strategies. Afterwards, stronger emphasis was placed on tractability 
and practicability aspects by applying parallelization and approximation techniques based on 
e.g. proximal decomposition [AOST 13a, AOS* 13b] or distributed agreement [CLLV14]. 


In [XZN13], the applicability of DMPC for a linearized model of interconnected power systems 
with different objective functions is shown. The authors in [FS12] propose a noncooperative 
DMPC method, which requires an information exchange about the agents’ planned trajectories 
via a non-manipulable information path. In addition, [RYNC14] proposes a noncooperative 
DMPC procedure, where each agent is equipped with a distributed Kalman Filter to estimate 
the overall state of the system. Both approaches are restricted to linear systems only. [LKF15] 
presents a contract-based DMPC method for agents with differing objective fuctions, which 
is applicable to nonlinear non-affine systems. However, it needs guaranteed information 
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about the future trajectories of coupling variables to be exchanged among agents. [VBGQ+20] 
presents a DMPC procedure for optimal dispatch of different network participants (in particular, 
conventional generators, RESs, and energy storage systems). However, the approach does not 
account for physical line limits and thus not prevent possible redispatch contingencies. 


However, as the strategic interplay of network participants is typically represented by a full 
peer-to-peer competition among players, it must be assumed that each has complete infor- 
mation of the other players’ strategies. Thus, a major drawback of classical noncooperative 
frameworks is their utterly high communication effort for broadcasting the relevant infor- 
mation in a non-discriminatory manner via non-manipulable information paths. Moreover, 
a calculation of the individual best response depending on each other player’s strategy is 
computationally demanding and poorly scalable, which makes a real-time solution intractable, 
except for very simplified academical case studies. 


In this context, Stackelberg approaches try to lighten this computational load by implementing 
a sequential sharing of information in a leader-follower manner. Stackelberg approaches are 
most common in the area of demand-side management. E.g. in [TZS*+14, YH15, MWH*18, 
EES* 19], a single-leader multiple-follower scenario is applied where a single ES is the price- 
setting leader and multiple consumers are the price-taking and utility-maximizing followers. 
In the area of VPP operation, [MR11] presents a Stackelberg model for the optimal interaction 
between a single VPP operator (leader) and multiple micro-cogeneration units (followers). 
Multiple-leader multiple-follower games are investigated in [ALCB15, LGCZ15, MMO17], 
allowing multiple price-setting ESs to compete on their part. However, due to its high 
computational effort, Stackelberg approaches require a time discretization with sufficiently 
large time intervals, thus there is a possible conflict between real-time computability and 
real-time adaptivity. Moreover, the underlying assumption that followers have no means at 
all to affect the leaders’ strategies seems rather disputable. 


A compromise between the fully decentralized information pattern of peer-to-peer competition 
and a centralized scheme is undertaken by price-based frameworks featuring different models 
of competition. Here, individual competition among players is replaced by a real-time bidding 
process against a single market instance. The two main directions studied in the context of 
power systems are Cournot competition and Bertrand competition, whose main distiguishing 
feature is which quantity constitutes the strategic decision variable. In Cournot competition, 
each participant bids quantities of power while respecting the given price per quantity. [NSH15] 
and [CBK17] present Cournot-based demand-side management schemes where a central 
control authority ensures supply-demand balancing. The authors of [DM19] propose a fully 
distributed pricing mechanism for a Cournot model of competition between price-setting 
generators and price-taking loads. In Bertrand competition, by contrast, the bid price is 
the strategic decision variable. Network participants send bids to an Independent System 
Operator (ISO), which evaluates the bids centrally and then dispatches them. Two recent 
papers [CC20] and [CSD* 20] develop a Bertrand competition model between price-setting 
generators resulting in a continuous-time bidding process against a centralized system operator, 
which is shown to provide economic efficiency. Transmission line congestion can be considered 
only for radial networks. In [SCD* 19], a Bertrand competition between generators and ISO 
was implemented by means of a time-triggered update scheme of bids and generator setpoints. 
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Another iterative algorithm is proposed in [MWY7*19] for interaction between multiple 
price-setting consumers and a centralized generation company. 


In spite of different terminology, e.g. ISO [CC20], macro-station [SHP11], central distributor 
[PLB* 16], non-profit central control authority [CBK17], these approaches follow the same 
principle: The centralized instance acts as the only counterpart and antagonist for every single 
player, with all information on the other players’ strategies being aggregated in the momentary 
price. In contrast to a generalized game-theoretic setup with peer-to-peer dependencies 
between all players, both models of competition provide better scalability, since the own 
individual strategy of each player depends only on the aggregate of the other strategies 
(Cournot) or on the strategy of the single ISO (Bertrand). This helps mitigate computational 
overhead, which is an advantage especially in large-scale power systems. 


Cooperative Game-Theoretic Approaches 


While noncooperative game theory supposes that no binding contracts are possible between 
players, cooperative game theory involves binding contracts between players and provides 
methods for allocating a jointly realized profit to the players involved. The two main branches 
in cooperative game theory are coalition formation and bargaining. 


Within coalition formation, the goal is to find stable alliances of network participants that 
result in an individual improvement for all as compared to not collaborating. In the field of 
coalition formation, a conceptual profit allocation strategy invoking coooperation among 
independent power producers is presented in [JY03], and a first coalition formation algorithm 
for radial distribution grids is presented in [SHP11]. Moreover, there has been considerable 
research effort into using the three classical solution concepts core, nucleolus and Shapley 
value, see e.g. [LH16, CBPt19], [HMM19], and [LH16, CK19], respectively. These concepts 
represent different manifestations of a stable equilibrium, originated by different notions of 
individual gratification, which aim to prevent each participant from leaving the coalition. 
While calculation of core and nucleolus is computationally expensive and possibly non-unique, 
the Shapley value, in contrast, can be calculated considerably easier. In [NKR13], a conceptual 
approach for arbitration of conflicting interests between producers and system operators based 
onacoalitional game is proposed. Through an additional power routing function, the approach 
is capable of preventing network issues such as transmission congestion. [FMC17] develops 
a discrete-time model for dynamic coalition formation of smart grid prosumers including 
storage devices. The last two approaches discussed also allow to account for transmission 
losses. However, their high computational cost prevents real-time application to large-scale 
systems. 


In contrast to coalition formation, bargaining approaches focus in particular on the sharing 
of the additional gains that exist as an outcome of cooperation. The classical minimum 
requirement for bargaining solutions is Pareto efficiency (cf. Definition A.5). Numerous 
papers deal with bargaining strategies among ESs, see e.g. [YTL12, FAP18, WH18]. In the 
area of power system dispatch, [WK09] presents a bargaining solution for small-scale DC 
power networks, where sources and loads may aim for different goals such as maximizing 
the own infeed of power or optimally following a certain voltage profile. However, since 
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players do not act simultaneously but sequentially, the method is unsuitable for large-scale 
systems. Other solutions for economic generator dispatch based on simple cost functions 
are presented in [WLM15] and [WWM16]. One central issue in all bargaining approaches is 
the strong correlation between simplicity of the model and computational tractability of the 
bargaining problem. In particular, finding a solution for nonlinear or nonconvex problems is 
computationally expensive even for low-dimensional problems. Consequently, most papers 
dealing with bargaining approaches for economic dispatch rely on highly simplified models 
of both the market and its participants. 


Generally speaking, approaches originating from cooperative game theory are subject to 
even more rigorous requirements e.g. with regard to revenue sharing mechanisms or multi- 
stage side payments. In practice, this implies the need for a centralized instance that has 
full knowledge about all states and control actions, and that is authorized to redistribute the 
profit over all players with respect to some predefined rules. However, this requirement is 
diametrically opposed to both operational and informational unbundling. 


2.2.3 Price-Based Congestion Management 


In the context of Cournot and Stackelberg approaches, it is shown e.g. in [YTN13, MWH*18] 
that the resulting real-time pricing controllers?® incentivize stakeholders to shift their genera- 
tion and consumption curves to off-peak hours and thus provoke a more balanced demand 
curve over time. However, a purely time-varying, but regionally uniform price is unable 
to reflect regional scarcity patterns since physical network constraints such as congestion 
cannot be internalized [TBW15, p. 212]. In particular, the methods discussed in the pre- 
ceding subsection do not protect against after-market reallocation by the system operator 
due to transmission congestion. A remedy to this situation is the introduction of locational, 
i.e. regionally differentiated, prices in order to adequately signal local network conditions. 
Moreover, locational pricing is considered to reduce structural and regional mismatches in 
generation and consumption by providing long-term investment signals to deploy generation 
capacity in load centers [Sta19, p. 29ff.]. Initial conceptual studies were conducted in [SCTB88] 
and [Hog92]. Afterwards, nodal pricing frameworks have been implemented in numerous 
countries, e.g. the US, Scandinavia and Australia. Although being highly efficient in avoiding 
redispatch, nodal pricing tends to support the exploitation pattern of BRPs (cf. Section 2.1.2), 
since large generators may exploit their own impact on their nodal price e.g. by temporarily 
cutting supply to provoke high local spot market peaks [BHJ16]. Accordingly, there is an 
ongoing debate among both policymakers and researchers about the most effective grain 
size of price zones’. From a practical viewpoint, it has been found useful to choose zone 
boundaries along transmission lines which are prone to congestion [Sto97]. 


Besides numerous research contributions on a rather strategic level, which typically analyze the 
long-term perspective, a few papers deal with real-time pricing schemes to address congestion 


20 In energy-related literature, the term real-time pricing is typically used for closed-loop pricing strategies, while 


time-of-use pricing refers to open-loop pricing strategies. 

The grain size of price zones varies widely across countries and states: While the price zones in Norway (5), 
Sweden (4) and Denmark (2) are supra-regional, there are 11 price zones in the state of New York and even more 
than 4000 in the state of Texas. 
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management. [MY16] and [MSM17] present centralized strategies for cost-efficient generator 
rescheduling based on an offline calculation of sensitivity factors. In [SCA16, SBCA19], 
active power flow limits are included in a centralized real-time economic controller, while 
[ZP15, ZLP15] present a distributed controller representation and [SCD* 19, CC20, CSD* 20] 
incorporate the limits in a real-time bidding scheme. However, all of the above pricing 
approaches rely on simplified network models such as DC-OPF or decoupled AC-OPF, where 
the crucial effects of transmission losses and reactive power flows on congestion are both 
neglected, cf. [Wei17, p. 220ff.; JB20]. 


2.2.4 Frequency Control 


In a power system fed by synchronous generators (SGs), deviations from the global active 
power balance cause a positive or negative mechanical torque on the rotating masses of the 
rotors? and thus a deviation of each nodal frequency from its nominal value. Vice versa, 
frequency control aims at restoring the nominal frequency by means of positive or negative 
active power injections. As of today, frequency control interventions are invoked by the TSO 
using a hierarchical sequence of actions based on the standard scenario of a large outage or 
load jump: Immediately after the failure, there is a change in the kinetic energy of the rotating 
masses and thus a steady decrease or increase in frequency. The rotational inertia prevents an 
immediate collapse of the grid, which is called momentary reserve. To stabilize the frequency, 
a decentralized proportional control (primary control) is activated within the first 30 s after 
the incident. Power plants participating in primary control are equally distributed over the 
interconnected grid. Using a specific droop coefficient, each power plant’s own positive or 
negative active power injection can then be calculated directly from in-situ measurements of 
the nodal frequency deviation. Since the proportional controllers from primary control lead 
to a steady-state frequency deviation, secondary control is activated in the next step in order 
to regulate the frequency towards its nominal value. For this purpose, each TSO calculates its 
own area control error by integrating the frequency deviation and then centrally activates the 
responsible power plants so that the required active power is available in no later than 5 min. 
Lastly, tertiary control (sometimes also called minute reserve), which is available within 15 min, 
aims at restoring the pre-contingency balance among TSOs by activating additional positive or 
negative power injections or, if necessary, re-scheduling the setpoints of the secondary layer. 
All three control layers are marketed separately on the BM” (cf. Table 2.1), with a further 
separation between positive and negative power for the secondary and tertiary level. 


With the ongoing transition from SG-dominated CPPs towards inverter-interfaced DERs, 
plenty of research has been undertaken to investigate how inverters can be seamlessly in- 
tegrated into the existing hierarchy of frequency control. The most prominent approach is 
inverter droop control, where the droop characteristic of SGs with proportional relationship 
between active power infeed and frequency deviation is mimicked. Originally published in 
[CDA93], this approach has since been extensively studied, e.g. in [TJUM97, CCG02, BPPt 08]. 


22 The dynamic behavior during the self-synchronization process between the nodal frequencies of an interconnected 


power system can be analyzed in more detail by the theory of coupled Kuramoto oscillators [DCB13]. 

Within the ENTSO-E supergrid, primary, secondary and tertiary frequency control are referred to as frequency 
containment reserve (FCR), automatic frequency restoration reserve (aFRR), and manual frequency restoration reserve 
(mFRR), respectively. 
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The main advantage of inverter droop control is its decentralized nature without the need for 
extensive communication and its full backward compatibility with a SG-dominated environ- 
ment. However, as with the conventional droop control for SGs, a steady-state deviation from 
nominal frequency remains. Besides, the practical effort to correctly tune the droop parameters 
is supposed to be high [CS13]. Apart from droop-based approaches, several alternative control 
methods have been proposed, such as virtual synchronous machine [BH07, DSF15], virtual 
oscillator control [SDJD15], or internal matching control [JAD16, MDS* 18], all with the goal of 
adopting the rotational inertia concept familiar from SGs to inverters. 


Owing to the faster operating time scales of inverter-interfaced DERs and new topological 
concepts such as MG, VPP, and WoC, numerous papers have investigated how to overcome 
the traditional hierarchy of primary, secondary, and tertiary frequency control by means 
of a unifying control strategy [GCLL13]. It was derived in [LCZL14] and [LZC16] that 
economic dispatch can be integrated into a unifying secondary control scheme. Moreover, 
in [DSPB16] it was shown that each minimizer of an economic dispatch problem can be 
reached by a decentralized droop controller with properly chosen droop coefficients. To 
combine the primary and secondary control level, the authors in [ADJS13, SPDB13] present 
distributed averaging proportional-integral (DAPI) controllers where zero frequency deviation is 
reached via a neighbor-to-neighbor communication path. A gather-and-broadcast approach is 
derived in [DG17], where the integration between economic dispatch and frequency control is 
performed in a semi-decentralized fashion with locally aggregated measurements ofthe nodal 
frequency deviations. In [FP10], primal-dual saddle-point flow is applied to frequency control. 
It allows to unify all three control levels together with economic dispatch by characterizing 
the desired equilibrium as a minimizer of a constrained optimization problem. Since the dual 
variables of the cost functions can be interpreted as price signals, no generation setpoints by a 
superordinate authority are needed. This understanding has led to the idea of continuous-time 
dynamic pricing controllers for frequency regulation [ZTL12, SDv15, TBD16, MZL17, SDv17a]. 
Furthermore, inequality constraints can be incorporated either by using extended Lagrangians 
[SDv17b], projection operators [ZP15, SDv17b, WLP* 19], or a saturation term [ZMLB18]. A 
rigorous performance analysis of primal-dual saddle-point flow applied to DC power flow 
can be found in [SPPMD16] and [CY17]. Detailed discussions of current research work on 
optimization-based frequency control are conducted in [DBSPG19]. 


Meanwhile, increased computing power has also permitted MPC-based and game-theoretic 
control methods to be considered for real-time or near-real-time purposes: A computationally 
tractable DMPC scheme for frequency control based on a linearized MG model is presented 
in [BT17]. The authors of [STY*18] present a DMPC strategy for heterogeneous power 
systems fed by CPPs and wind power plants, which is able to achieve frequency restoration 
by incorporating ultra short-term wind forecasts. In addition, there exist noncooperative 
concepts to frequency regulation which address the individual goals of stability providers 
[CYWL15, NOS*15, CSP*19]. While [CYWL15] focuses on load frequency control using a 
linear-quadratic differential game, [NOS*15] provides a real-time framework for frequency 
control by means of time-varying prices offered by a centralized ISO. Both approaches rely on a 
linearized system model. In [SCA16], an aggregate frequency error is used as additive feedback 
signal in a real-time WEM. In [CSP* 19], advanced meta-heuristic optimization techniques are 
applied to a nonlinear system model in order to improve the frequency response characteristic 
on device level. 
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2.2.5 Voltage Control 


Voltage control aims to keep voltage magnitudes at each node within a tolerance band around 
the specific nominal voltage. Unlike frequency control, local actions of voltage control only 
have a local impact, thus control actions are typically?* accomplished in a three-stage process 
with increasing geo-spatial coverage: At the lowest level (primary control), the local voltage 
is regulated to a given setpoint. For this purpose, excitation systems of the SGs (automatic 
voltage regulation) as well as active and passive network components, e.g. phase shifters 
and flexible AC transmission systems (FACTSs) are applied. At the secondary control level, 
local setpoints within an entire zone are adjusted so that a designated pilot point matches 
its setpoint and, in addition, there is no voltage band violation at any node. Lastly, tertiary 
control aims to centrally coordinate the individual zones and adjust the setpoint voltages at 
pilot points. 


To enable seamless integration of inverter-inferfaced DERs into the existing control hierarchy, 
numerous decentralized droop-based approaches for voltage control have been proposed 
[CDA93, CC18, LSZ18]. Here, local deviations from the voltage setpoint are compensated by 
additional reactive power injections similarly to the active power injections of the frequency 
droop approach. Another similarity to frequency droop is the nonzero steady-state deviation 
due to the purely proportional control architecture. Namely, it can be shown that in order to 
keep the steady-state deviation within an acceptable range, R/X ratio of all lines must be small, 
ideally zero. Yet, this assumption is not admissible for distribution networks [MMMT12]. 


There are several alternative methods for voltage control based on the formulation of optimiza- 
tion problems. As shown in [FCL13], the standard voltage droop control is inversely optimal 
to a convex optimization problem. Based on this finding, [ZC16] and [KZGB16] propose an 
iterative procedure for adjusting the droop characteristic in order to keep all voltages of a 
radial network within an acceptable range. [SPDB17] presents a quadratic droop controller as 
the solution to an optimization problem which minimizes a trade-off between voltage setpoint 
deviation and reactive power dissipation. [HGK18] presents an approach how to optimally 
tune the droop parameters. 


Furthermore, several publications have appeared in recent years documenting distributed 
non-droop-based methods with neighbor-to-neighbor communication among nodes. [CTFS18] 
proposes a combination of sliding mode and DAPI control, where the weighted average voltage 
of the MG is provably equal to the weighted average of all voltage references. [THL19] develops 
an optimization-based procedure for voltage control in radial distribution grids, where the 
objective function reflects the setpoint deviation of voltages and where upper and lower limits 
on reactive power injection are specified by box constraints. The proposed method is based 
upon the assumption of a uniform R/X ratio throughout the network. In addition, [LSZ18] 
allows to specify the overall amount of reactive power injection as additional soft constraint 
and a linearized relation between voltage and reactive power as additional equality constraint. 
To enable an online computation, [THL19] uses the distributed accelerated dual descent 
algorithm, while [LSZ18] is based on the alternating direction method of multipliers. Both 
approaches rely on a linearized system model. 


24 The degree of automation of each level as well as the responsibilities of TSO and DSO highly depend on the 


specific grid code. The interested reader is referred to [vV08, MC11] for further elaborations. 
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In the context of MGs, several authors propose passivity-based approaches to voltage stabi- 
lization. In [ABEPCR18, ABMEPG18], primary voltage controllers are developed on the basis 
of a linearized port-Hamiltonian model” of the MG, such that a specific energy function of 
the closed-loop system has its minimum at the desired setpoint voltages. [AvJ17] presents a 
secondary voltage controller based on a nonlinear port-Hamiltonian model which provides 
zero setpoint deviation in lossless networks. 


More recently, a number of game-theoretic methods addressing voltage stability have emerged. 
Most papers focus on sequential approaches, where a trade-off between the grid-forming 
generators and, if applicable, consumers can be found either iteratively [GA20] or in a leader- 
follower manner [CZ18]. A simultaneous approach is proposed by [WHK18], where voltage 
stabilization is achieved by incentivizing PV owners with additional financial payments, if 
they follow a given desired voltage profile. Yet, the proposed method is applicable for radial 
networks only. 


2.2.6 Combined Frequency and Voltage Control 


Regulating frequency and voltage separately from each other always relies upon the assump- 
tion that line resistances (or line reactances”°) are negligible. However, especially in medium 
voltage networks, which are neither dominantly inductive nor dominantly resistive, this as- 
sumption does not hold in practice. For this purpose, [GMG* 07, LK09] use virtual impedance 
techniques which allow a modification of the setpoint voltages by means of a second feedback 
loop in order to compensate the cross-term. In [RLBR12, BS13], a generalized droop control 
is introduced, which provides coupled droop characteristics with cross-terms for active and 
reactive power. With the resulting equations, it is required that all line and load parameters 
are known. Moreover, similar to decoupled droop control, generalized droop control cannot 
regulate the steady-state deviation of frequency and voltage to zero. 


In [SPSD* 15], a DAPI controller for primary and secondary frequency and voltage control in 
inverter-based MGs is proposed. While the primary level relies upon a droop characteristic, 
the secondary controller uses a consensus protocol in order to restore the nominal frequency 
by means of an arbitrary connected communication path between the agents. Moreover, the 
presented controller in [SPSD* 15] provides a tuning functionality for the conflicting goals of 
reactive power sharing and voltage regulation. [DMSD16] augments the DAPI controller by 
a modified integral control law to minimize a quadratic optimization problem for the active 
power generations. In order to achieve better reactive power sharing, [WSI18] combines a 
consensus protocol for secondary frequency and voltage control with an additive correction of 
the voltage references. A slightly different approach is presented in [SGV14]. Here, distributed 
consensus is reached via all-to-all communication among all agents in the network. The 
approach is capable of providing both frequency and voltage regulation without knowing the 


25 The port-Hamiltonian modeling framework [vJ14] is a powerful modeling tool for complex multi-domain physical 


systems that has gained increased attention from various fields of application in the last years [MMS09, FZO+ 13, 
AS17, ACRMA19, PCH* 20]. A brief introduction to the basic terminology used in the context of this dissertation 
is given in Appendix A.4. 

If the network is assumed to be purely resistive, then frequency and voltage are decoupled as well, where 
frequency can be controlled by reactive power and voltage by active power [LSK05]. 
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physical network parameters. However, both the amount of tunable parameters as well as 
overall communication effort is substantial. 


Among non-droop-based methods, a number of contributions employ passivity theory to 
obtain frequency and voltage controllers with zero setpoint deviation: In [RSFT15, TFRFT16, 
TFT17], stabilizing controller parameter are found via a centralized solution of linear matrix 
inequalities. An extension is presented in [NFT19], which allows to include the dynamics of 
the power lines into the underlying MG model. In [SMKH19], the criterion for stabilizing 
controller parameters is further simplified to sufficient conditions which can be calculated 
a priori. Besides, [DE18] presents a first differential game formalism for joint frequency and 
voltage control of inverter-based islanded MGs. However, as pointed out by the authors, a 
high computational effort as well as scalability issues limit its applicability to small-scale 
systems only. 


2.3 Discussion 


In light of the future developments and disruptions, the preceding section conducted an 
extensive review of current research trends towards a stable, economically efficient and grid- 
supportive operation as objected at the end of Chapter 1. Table 2.2 provides a summarizing 
overview of existing papers on distributed control methods that combine at least two of the 
subjects economic dispatch, frequency control, or voltage control. 


First, we analyze the adequacy of the state of research in ensuring a stable operation of future 
power systems. Table 2.2 indicates that research in voltage control has tended to concentrate 
on internal and primary control methods specifically suited for inverter-based networks, and 
to overlook integrating secondary and tertiary voltage control strategies from an optimization 
viewpoint. Since the traditional control hierarchy described in Section 2.2.5 is typically kept 
untouched, it has to be assumed that a higher-level controller is able to provide optimal 
setpoints. Although a few papers, e.g. [LQD14, KZGB16, GA20, HBHD21a], have addressed 
the question of autonomous voltage control frameworks that unify the primary and secondary 
control layer, all of these approaches are subject to a centralized controller and thus exhibit a 
single point of failure. 


In the area of frequency control, the rise of intermittent DERs and the new phenomenon 
of volatile residual loads have led to a new, more general understanding of the traditional 
hierarchy and its purpose: Originally defined as a scheduled cascade which has to be activated 
in a strict sequence in the event of a major power plant outage, the literature nowadays 
more generally defines primary control as frequency containment and secondary control as 
frequency restoration, which has led to many unifying control approaches in which both goals 
are addressed at the same time. Meanwhile, a strong connection between tertiary frequency 
control and economic dispatch has been exposed [LCZL14], which later resulted in the idea of 
real-time dynamic pricing. 


However, it can be seen from Table 2.2 that all previous papers dealing with the combination 
of economic dispatch with frequency and voltage control are not capable in providing nonzero 
deviation of frequencies and voltages from their respective setpoints. This limitation is caused 
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Table 2.2: State of literature on multi-disciplinary real-time control methods for interconnected power systems. 
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by the fact that the controllers are designed under the simplifying assumption that line 
conductances can be neglegted. Yet this assumption, as mentioned earlier, is inadmissible 
especially for distribution grids [MMMT12]. As a consequence, a practical implementation 
of all of these controllers would still require some kind of additional frequency or voltage 
restoration by a superordinate controller. 


Moreover, it was pointed out in e.g. [Sch15] that the heterogeneity of future power systems 
requires that frequency and voltage control have to be provided by both conventional SMs 
and inverter-interfaced DERs. However, referring to the first column of Table 2.2, the existing 
controllers are applicable only to a limited group of network connectors, for instance con- 
ventional SMs or grid-forming inverters only. These findings together expose the following 
research gap. 


Research Gap 1. For interconnected power systems with nonzero transmission losses, 


there are no control strategies for optimal dispatch of heterogeneous network connectors 
which ensure both frequency and voltage stability. 


With regard to the second sub-objective of economic efficiency, the existing controllers reveal 
several weaknesses: As Table 2.2 suggests, most control approaches follow a common goal, 
i.e. all network participants are assumed to be non-competitive. On the contrary, price- 
based approaches such as [NOS* 15, SCD+19, CSD* 20], which allow for individual profit- 
maximizing agents, feature a centralized information and communication structure, with a 
single node that is responsible for the entire pricing process. The few existing approaches 
with both competitive agents and distributed communication all require to disclose their own 
strategy to all other competitors. Thus, no existing approach is able to fully amalgamate 
both market and control strategy while also achieving legal, operational, and informational 
unbundling. Even worse, due to their higher communication effort, these approaches are 
likely to further amplify the interferences and dependencies between competitive electricity 
providers (CPPs or inverter-interfaced RESs) and regulated infrastructure providers (system 
operators), that are already occuring at present (cf. Section 2.1.2). 


Besides these structural inefficiencies, the employed game-theoretic solution concepts of 
Nash [SLLO2, MWY* 19], generalized Nash [AOS*13b, AOS* 13a], or Stackelberg equilib- 
rium [TZS+14, ALCB15, EES* 19] only indicate the existence of some equilibrium, which 
no participant will unilaterally leave. In particular, these solution concepts are not able to 
properly assess whether the resulting equilibrium might be dominated by another equilibrium 
which is more desirable from the participants’ points of view (i.e., generates higher profits for 
some of the participants). 


As a further major drawback, most market-based control approaches also rely on very sim- 
plified models with zero line resistances. Due to the disregard of losses over transmission 
lines, the resulting market-clearing prices are infeasible, i.e. this again requires some sort of 
after-market mechanism with an exchange of side payments and/or additional grid tariffs, 
which would have had to be supervised by a higher-level authority. Therefore, to the best of 
the author’s knowledge, all existing price-based control schemes are unable to balance the 
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resulting net payments to zero at market clearing, whenever the underlying power network 
has nonzero transmission losses. 


Research Gap 2. There is no systematic control framework which provides a trade-off 


between individual interests and market balancing while complying with the desirable 
information barriers between power plants and system operators. 


For the third sub-objective of grid-supportive operation of power systems, the existing price- 
based control methods suffer from a number of pitfalls that prohibit to incentivize generation 
close to load centers. In particular, the vast majority of price-based approaches developed 
so far result in temporally different, but geographically uniform prices. Therefore, it has 
no spatial control effect in any case. On the other hand, iterative bidding approaches such 
as [SDv19, CC20], which allow for geographically non-uniform prices, do not account for 
transmission losses throughout the network. In this way, the copper plate assumption, which 
was problematized in Section 2.1.2, persists, thus there is no incentive for local energy supply 
near load centers (cf. [HSMT19, p. 14]). Likewise, the conceptual approaches for nodal and 
zonal pricing surveyed in [Wei17] always rely on simplifying DC-OPF models not considering 
transmission losses, which are not able to accurately reflect the actual states and physical 
relationships in the network, especially if the network is heavily loaded [Bak21, p. 266]. 


Considerably, no price-based control approach is able to combat congestion in real time. The 
majority of congestion management strategies are on a larger time frame, e.g. with time 
slices at an hourly basis and a daily planning horizon. With regard to real-time approaches, 
Table 2.2 shows that several authors integrate congestion management strategies as additional 
constraints into the economic dispatch problem. However, in most cases this economic dispatch 
results in a centralized optimization problem which is solved by non-competitive agents. The 
few competitive approaches considering congestion management [SCD*19, CSD*20] are 
only applicable for power systems with a tree topology. Moreover, all of the existing real-time 
congestion management strategies neglect the effect of reactive power flows on network 
congestion. At the same time, none of the purely competitive approaches with distributed 
communication (last three rows in Table 2.2) are able to properly reflect network congestion. 


All in all, existing dynamic pricing controllers fail to implement a real-time mechanism in 
order to incorporate network effects to the local prices. Within the literature, there is a 
large gap between long-term, investment signaling schemes, which are purely conceptual 
and do not provide a real-time mechanisms, and real-time pricing controllers that neglect 
the physical characteristics and limitations of the network and thus provide no means for 
incentive signaling. Therefore, a holistic short-term control framework is lacking, that is able 
to hold grid participants accountable for their own impact on the momentary state of the 
network. 


Research Gap 3. There exists no constructive pricing mechanism to provide regional 
market incentives for power plants by evaluating the momentary positive or negative 


impact of local power generation. 
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Figure 2.3: Outline of main contributions. “Higher-level” contributions on slower timescales are positioned at the 
top, while “lower-level” contributions acting on faster timescales are positioned at the bottom. 


2.4 Statement of Contributions 


To overcome the limitations and gaps of existing methods, this dissertation is dedicated to 
developing a distributed, continuous-time control framework for competitive power systems 
based on real-time dynamic pricing. By consistently accounting for transmission losses, the 
resulting control scheme shall not require any kind of settlement payments or after-market 
clearing procedure. Furthermore, due to a rigid compliance with information barriers be- 
tween economic players and system operators, the presented brownfield mechanism shall be 
incentive-compatible in a fully unbundled environment. All control laws are expressed as ex- 
plicit differential equations. The main contributions of this dissertation and their relationships 
to the revealed research gaps are illustrated in Fig. 2.3 and briefly outlined in the following. 


Contribution 1. This dissertation develops a distributed, market-based feedback control 


strategy for lossy and heterogeneous power grids, incorporating both, frequency and voltage 
control, with optimal dispatch. 


The control methods presented in Chapter 3 of this dissertation allow a real-time optimization 
of the connected DERs as well as conventional SGs, with the primary goal of maintaining 
the nominal frequency and keeping all voltage magnitudes within predefined limits whilst 
minimizing an individual user-defined cost function. The proposed controller is unifying in 
the sense that the former hierarchical division into primary, secondary, and tertiary frequency 
and voltage control tasks are combined within a single controller (see bottom panel in Fig. 2.3). 
Furthermore, frequency and voltage stabilization is maintained simultaneously. The resulting 
market-based control structure is fully compliant with legal, operational, and informational 
unbundling. In particular, motivated from an economic perspective, no communication 
between competitive players, no generation of exogenous setpoints by a higher-level authority, 
and no disclosure of individual cost functions is necessary. The controller design is based 
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on a nonlinear, network-preserving model?’ 


implementation. 


of the power system, and allows for a distributed 


Contribution 2. This dissertation presents a dynamic pricing framework for vertically 


unbundled power systems that enforces Pareto efficiency and market-clearing allocations 
among competitive participants. 


The real-time pricing approach presented in Chapter 4 (see middle panel in Fig. 2.3) enables 
each network participant to selfishly perform an individual profit maximization by means of 
internal control actions. In the light of [Ahl18, p. 324f.], the game-theoretic notion of Pareto 
efficiency (cf. Definition A.5) is employed to ensure economically efficient operation of all 
network participants. Accordingly, the resulting closed-loop equilibrium is such that there 
exists no other equilibrium where one or more network participants take a higher profit. A 
distributed market balancing scheme ensures that the total revenues of producers are always 
kept identical to the payments of consumers. This permits the additional costs arising from 
momentary transmission losses to be reflected directly in the real-time price, without creating 
net deficits in overall generation or an accumulation of capital. The resulting (optimal) market 
balancing problem gives rise to the problem class of (dynamic) optimal control problems and 
differential games subject to port-Hamiltonian systems (PHSs) (cf. Appendix A.4), for which no 
solution concept in the sense of an explicit stabilizing control strategy exists yet. 


Contribution 3. This dissertation develops a continuous-time optimal control method for 


(nonlinear) port-Hamiltonian systems with N agents. 


In Chapter 5 of this dissertation, we propose a continuous-time adaptive control strategy for 
solving single- and multi-player noncooperative differential games with general (Lagrangian) 
performance indices and system dynamics modeled as general nonlinear input-state-output 
PHSs. The proposed control law (see right panel in Fig. 2.3) implements an online learning 
procedure which uses the Hamiltonian of the system as initial value function candidates. 
The feedback strategy of players is extended by adaptively weighting the individual value 
functions to ensure convergence to the Nash solution. Necessary and sufficient conditions 
for stability of the resulting controlled system are provided by employing Lyapunov stability 
theory. Since PHSs have received substantial attention as being a powerful tool for systematic 
modeling of dynamical multi-physics systems, the presented methodology qualifies for a 
broad range of applications apart from power systems. 


Contribution 4. This dissertation presents feedback control strategies for incentivizing 


grid-supportive behavior of power plants by continuous generation of price signals. 


In order to contribute to a grid-supportive mechanism design, this dissertation provides a 
distributed framework for real-time zonal pricing. Being fully compatible with the WoC 


27 A power system model is said to be network-preserving if each bus is modeled separately (possibly as a dynamic 


subsystem) and connected to the other buses via power flow equations. Conversely, the model is said to be 
network-reduced, if buses within a certain geographical region are aggregated and treated as a single bus. 
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concept introduced in Section 2.2.3, local CCs are enabled to allocate cell-specific, temporally 
and spatially differentiated prices. Thereby, higher-level goals such as congestion management 
or local investment signaling can be implemented in both the short term and long term (see 
top panel in Fig. 2.3). In fact, the presented mechanism allows to pursue those equilibria that 
are desirable from a global perspective by means of a distributed consensus protocol. 


The above contributions are systematically elaborated in the subsequent Chapters 3-5, which 
are organized as follows: Chapter 3 develops a network-preserving power system and market 
model conforming with the WoC concept, and presents a price-based control framework 
with distributed intra-cell communication to satisfy Contribution 1. Chapter 4 presents a 
distributed control framework at inter-cell level for an automatic regulation of spatial price 
differences, working towards Contribution 2 and Contribution 4. Finally, Chapter 5 provides 
the solution of dynamic optimal control problems for PHSs to complete the remaining tasks 
of Contribution 2 along with Contribution 3. 


3  Optimization-Based Control of Cellular 
Power Networks 


In this chapter, aWoC-based control framework for economic dispatch and power system 
stabilization of competitive network participants acting in an unbundled power system is 
developed. After introducing the general setup of the WoC-based infrastructure in Section 
3.1, Section 3.2 presents a dynamic, network-preserving model of the physical power system. 
Section 3.3 investigates the competitive behavior of unbundled network participants and 
develops a conforming market mechanism. Section 3.4 derives price-based local controllers at 
intra-cell level and investigates the overall system dynamics resulting from the coupling with 
the physical system. Thereby Contribution 1 is provided. Section 3.5 examines the interaction 
between cells and gives an initial diretion towards inter-cell pricing strategies. Section 3.6 
briefly discusses the main findings of this chapter”. 


3.1 General Setup 


We consider an interconnected power system with meshed physical topology, which is modeled 
as a directed graph 9, = (Vp, Ep). The nodes v € Vp correspond to np buses and the edges 
e € E, to m, power lines connecting the buses, where the direction of the edges indicates the 
direction of positive power flow. The incidence matrix (cf. Definition A.10) of % is denoted by 
Dy. The power system is divided into nz € N cells operated by a CC k € Z = {kı,ka,...}. 
Within its own cell boundaries, each CC acts as an ISO, being exclusively responsible to ensure 
the main objectives of stability, economic efficiency and grid-supportive operation outlined in 
Chapter 1. The responsibilities and information patterns assigned to the CCs as well as their 
interactions with other network participants are detailed in Section 3.3. 


Regardless of cell boundaries, the power system is connected with wholesale prosumers (WPs) 
and retail prosumers (RPs). Both WPs and RPs are generic groups for network participants that 
produce and/or consume electrical energy. A major difference between both groups, however, 
is the presence or absence of flexibility: For (flexible) WPs, the own power supply or demand 
is the result of economic considerations (i.e. maximization of own profit) and thus supposed 
to be affected by prices. For (inflexible) RPs, on the other hand, the supply or demand of RPs 
is assumed to be uncontrollable. These two classes thus cover all possible types of energy 
producers and consumers. 


8 Preliminary versions of the results of this chapter have been published in the conference papers [KBKH19, 
KDB*+ 20, KWKH20]. 
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For the overall model, let W = {71,72,...} denote the set of ny WPs”. Each node is 
associated to at least one WP m € W. We denote the subset of nodes i € Vp associated to 
WP m € W by Vw, C Vp. Moreover, each node i € Vp belongs to exactly one of the nz 
cells Vz p C Vp, k =1,...,nz, ie. Vz k N Vz, = Í for all k Al and U;,2, Vz,r = Vp. The 
edges in €, can further be divided into the set Ep of inter-cell or boundary edges representing 
power lines that connect buses located in different cells” and the set E"P of intra-cell or non- 
boundary edges representing power lines that start and end within the same cell. The physical 
interconnection of different cells is described by the graph G, = (Z see) with incidence 


matrix Dp- 


A fundamental principle of the WoC concept is that measurement and control information is 
only shared locally. This is expressed by a separate communication infrastructure for each 
cell k € Z represented by directed graphs sk = (Vz,k, Eck) with incidence matrices D..x. 
Note that the direction of the edges of the communication infrastructures do not imply a 
one-way connection but rather the direction of positive “communication flows”. The following 
assumption holds under mild conditions: 


Assumption 3.1. The graph of the physical system Y, and all communication graphs 


Gon, k € Z are weakly connected. 


Assumption 3.1 implies that G, is weakly connected as well. 


The totality of all communication between nodes is represented by the communication graph 


Go = | Vo U Ecx | - (3.1) 


keZ 


This leads to the following formal definition: 


Definition 3.2 (Cell). A cell is a weakly connected component in s. 


Notably, the characteristics of the WoC scheme rely only on the communication topology, 
while the physical topology is not subject to any restrictions. In particular, power flows across 
the cell boundaries are allowed without any limits. In the following, we summarize the main 
properties and assumptions on the considered power system: 


Definition 3.3 (Competitive Power System with Cell-Based Infrastructure). We 
consider an interconnected power system with the following characteristics. 


e We consider a lossy AC system with meshed topology, 


29 Note that this framework is compatible to the VPP concept (cf. [NY21]), since the individual resources of 


prosumers may be located at geographically distant nodes in different cells. 
3° Tf there are multiple inter-cell power lines connecting the same cells, then Ep contains an edge for each of these 


power lines. 
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zu Be 


(a) WoC topology (b) MG topology 


Figure 3.1: Differences between WoC and MG topology. 


e the topology (i.e. the interconnection by power and communication lines) is fixed 
during the examination period, 


e there may exist multiple points of coupling between cells, 
e WPs and/or RPs are distributed arbitrarily through the power system, 


e the generation capacity available in the cell is not necessarily sufficient to cover its 
own demand, 


e there exists no superordinate network layer. 


Remark 3.4. Compared to our model from Definition 3.3, the definition of a grid of MGs [Sch15, 
Definition 3.2.1] differs by the following limiting characteristics: 


e For each MG there exists a single point of common coupling to a superordinate grid layer, 


e each MG can switch from islanded to grid-connected mode. In grid-connected mode, it be- 
haves as a single controllable generator or consumer from the viewpoint of the superordinate 


grid, 


+ it possesses enough capacity to supply most of its loads autonomously. co 


Remark 3.4 emphasizes that the MG concept is subject to more structural restrictions as well as 
more conservative capacity requirements (which are hard to verify in advance) than the WoC 
concept pursued in this work. Fig. 3.1 schematically summarizes the topological differences 
between these two concepts. Using the above notation, the topology of an islanded MG is 
regarded as the special case an = (). In the following, however, we will generally assume 
that Y A 9, and Ês # Ú hold in order to permit a global exchange of power. For reasons of 
comparison, though, the MG approach will occasionally serve as a reference scenario. 


3.2 Modeling of the Physical Network 


The set of nodes can be partitioned as Vp = Vs U Vz U Ve, where each subset represents a 
certain type of network connectors, namely, synchronous machine-type (S) nodes, inverter-type 
(T), and load-type (£) nodes. S nodes are connected to synchronous machines (SMs) of e.g. 
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Table 3.1: Network connectors. 


No. of Independent Dependent 
Nodes Controllable Uncontrollable 
S ns Deis Ufi Pei 0i, qi 
T NT Deis Ui Dei 0i, qi 
É ne Peis Ui 0i, Ui 
Total No. Np 2Np 2np 


gas or hydro turbines. Z nodes are connected to power electronics interfaced DERs. £ nodes 
are connected to producers or consumers with a fixed active and reactive power supply or 
demand. Sand Z nodes 2 € Vs U Vz are each equipped with an active power generation input 
Pg, and an active power consumption input py ;. £ nodes i € V; are equipped with active 
and reactive power consumption inputs pe ; and ge ;, respectively. Table 3.1 gives an overview 
of the three node types along with their specific independent (exogenous) and dependent 
(endogenous) variables. The set N, ; denotes all neighboring nodes j € Vp of i € Vp with 
(i, j) E€ Ep or (j,i) € Ep. 


Remark 3.5. S and T nodes correspond to the notion of PV nodes from classical power flow 
calculation. In the presented framework, the voltage magnitudes are either controlled directly 
(at T nodes) or indirectly via the excitation voltages U; ; (at S nodes). £ nodes correspond to the 
notion of PQ nodes, where the active and reactive power generation or consumption is given and 
the voltage magnitudes and bus voltage angles 0; follow from Kirchhoff’s laws. 


In classical power flow calculation, apart from PV and PQ buses there typically exists one slack 
bus 7 € Vp, which represents a large power plant or a superordinate network with infinite capacity 
of real and reactive power. The slack bus is characterized by fixed U, and 0; as independent and 
pi and q; as dependent variables. As the resulting power flows are invariant with respect to a 
uniform shift of all bus voltage angles 0;, the slack bus typically serves as bus voltage angle 
reference by setting 0; = 0. Yet in line with Definition 3.3, our model is intended to avoid any 
external sources of power. Thus, no slack bus is considered within our power system model. 
However, to provide for nonsingular solutions within calculations, all CCs and prosumers use bus 
voltage angle differences 3;; = 0; — 0; instead of explicitly dealing with the absolute values 0; 
and 0j. © 


The following subsections systematically derive a common model for the physical interaction 
of the power system. Subsection 3.2.1 considers the power flows over lossy transmission 
lines. Subsections 3.2.2-3.2.4 present an appropriate model for SM-type, inverter-type, and 
load-type nodes, respectively. On this basis, Subsection 3.2.5 synthesizes the dynamic model 
of the overall physical plant system. 
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3.2.1 Power Lines 


Regarding the physical infrastructure, Definition 3.3 does not restrict the voltage level of the 
underlying power grid. In the following, however, we will focus on medium voltage networks 
(with nominal voltages typically between 10 kV and 100kV), as the majority of DERs are 
connected at this voltage level. In addition to the properties from Definition 3.3, medium 
voltage networks can be assumed to have the following characteristics under normal operating 
conditions. 


Assumption 3.6. a) The network is operating around the nominal frequency. 


b) The network is a three-phased AC system which is operated under symmetrical 
conditions. 


Note that Assumption 3.6a) is only needed for the purpose of model validity and does not 
imply the (much stricter) requirement of frequency stability to be addressed in the following 
sections. Assumption 3.6b) implies that the network is both symmetrically configured and 
symmetrically operated, see [SFO15, Section 2.4.1] for basic terminology. With Assumption 
3.6b), a single-phase equivalent circuit model [MBB12, p. 66] can be applied (see Fig. 3.2). Here, 


Figure 3.2: Infinitesimally short line segment of length dh. 


each phase of a power line connecting two nodes i, 7 € Vp is decomposed into an infinite 
number of increments dh (see Fig. 3.2). Each of these increments can be represented by a 
specific series resistance R;; characterizing the thermal losses, a specific series reactance Xj, 
characterizing the flux linkage within the power line and the flux linkage to the other phases, 
a specific parallel susceptance B;, describing the potential difference between the individual 
phases, and a specific parallel conductance G‘,,. 


With Assumption 3.6b) and provided that no line length exceeds 300 km, the Il-equivalent 
circuit model can be applied [And, p. 101; MBB12, p. 67]. It provides a lumped-parameter 
representation (see Fig. 3.3) with 


= DL. — 
Zu = y Sinh (ADI (3.2) 


YLig = 


pa 


(3.3) 
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Zus 


Figure 3.3: II-equivalent circuit model for a power line between nodes i and j. 


where Z, = Ri, + 3Xi;j and Y} = Gij + 9Bi;- Together with the nodal shunt admittance 
Ysh,i = Jsh,i+Jbsh, i, we obtain Y; = Jenit jen, ; YL ij. Consequently, the entire network of 
power lines can be described in vector-matrix notation by the nodal network equation [MBB12, 
p. 113] 


T,=Y,U> (3.4) 
with T, = colev, {I;} and Ü, = colzev, {Ui}. The nodal admittance matrix Ý, describes 


the relationship between the nodal voltages Ü, and the currents I p entering the nodes. If 
node 7 is connected to node j by a line, then 


2 2 1 
Y | = [¥ | Zee 3.5 
| Plaj Pla Zia pa 
e 3 1 
Yo] =#+ E = (3.6) 
ii jEN pi Ly, ij 


Using the nodal admittance matrix ys, the apparent power balance can be determined to 


BEN. ÖY, (3.7) 


iEVp GENS A 


where 5; denotes the net power flow from node i € Vp to neighboring nodes. Now the 
equation can be split into its real and imaginary part [SFO15, p. 90]: 


m=), 2, Vlies; — Bij), (3.8a) 
iEVp JENS, i 

q=) >» ViVi p.ig sin(vij — Bis), (3.8b) 
iEVp JENp,i 


where Yp ij := IY playl and bij := are([Y plij)- 


A widely used representation of the power flows (3.7) are the hybrid network equations, where 
the nodal admittance matrix is given in rectangular coordinates using Yp =: Gp + JB»; 
while the nodal voltages are given in polar coordinates using Ü=:Uo exp(j0). This yields 
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the network equations at node i € Vp as 


Pi = U? Giu + 3 U,U; (Bi; sin(d;;) + Gij cos(d;;)) 5 (3.9) 
JEN», 
qi = U? Bi + 5 U,U; (Gij sin(d;;) = Bi; cos(d;;)) z (3.10) 
jENp,i 


The (sending-end) power flow from node i € Vp to node j € V, is given by Š = Pij + IQij 
where 


Pi; = —U? Gij + U;iU;Gij cos(d;;) + UU; Bi; sin(vi,), (3.11) 
Qij = U? Bij + UU; Bij cos(d;;) + UU; Gi; sin(V;;). (3.12) 


Hence, the active transmission loss over line (i, j) € Ep equals 


®ij = Pij + Pi = —(U? + U2) Giz + 2U:U;jGij cos(ðij). (3.13) 


3.2.2 Synchronous Machine-Type Nodes 


SMs form the backbone of today’s power systems. Besides their main application of converting 
mechanical energy into electrical energy, e.g. as turbogenerators in bulk CPPs, they can also 
serve as constant-speed drives, e.g. for centrifugal pumps or air compressors. 


The constant magnetic field in the rotor (typically governed by a DC exciter winding) interacts 
with the rotating field of the stator by inducing a rotor voltage (or polar wheel voltage) in 
the stator windings, where in motor mode the rotor follows the stator rotating field, while in 
generator mode, the stator rotating field follows the rotor. The rotor field winding of the SM is 
aligned with the direct (d-) axis, while the axis perpendicular to the d-axis is called quadrature 
(q-) axis. Both axes form a reference system inside the machine that rotates with the rotor. 
Under Assumption 3.6a), the electrical side of the SM at node i € Vs can be adequately 
described by the sixth-order model (3.14)-(3.16) from [MBB12, p. 455] 


Das = UG 4 Rsa = X lau (3.14a) 
Ugq,i = Ui F XG alai — Rs ilq,» (3.14b) 
T5060 = Up — Uy, + Iai: (Xani — XG); (3.14c) 
76404 =- Wit), (3.14d) 
90,02: = Uae Uki + las (Xaa — Xd), (3.14e) 
To dg = Uha — UN + Taya (Xha — XY), (3.14f) 


where U’, and U’, denote the subtransient and transient electromotive forces (internal 
voltages) induced at the stator coils and U,.., denotes the terminal voltage of the SM. Due to 
time-varying screening effects in the rotor, usually a distinction is made between subtransient 
state, transient state, and steady state, which is represented in (3.14) by the (sub-)transient 
(+) Z Tas each 


and steady-state reactances X () , and time constants Tjo; < TY, and Ty 


q0,i 
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modeled separately for d- and q-axis. For a detailed derivation of the model (3.14), the reader 
is referred to the comprehensive textbooks [Kun94, BV00, MBB12]. 


The mechanics of the rotor is described by a second-order system following from Newton’s 
equation of motion 


Ji = Omi, (3.15) 
an = Das Om; = Mri 75 Mes; (3.16) 


where y; denotes the rotor angle between the q-axis and a static (non-rotating) reference frame, 
J; is the moment of inertia of the turbogenerator set, Dg; is the damping-torque coefficient, 
Mr, is the turbine torque, and M, ; is the (counteracting) electromechanical torque. 


We make the following simplifying assumptions for the SM model. 


Assumption 3.7. a) Subtransient dynamics of the SMs are neglected. 


b) The screening effect of the rotor body eddy currents in the q-axis and the stator 
resistance is neglected. 


c) Saliency of the rotor is neglected. 


Assumptions 3.7a) and 3.7b) are justified by the timescale limitation adopted in Section 2.2 
(cf. Fig. 2.2). Assumption 3.7c) reflects the most common configuration of turbogeneraors in 
CPPs. 


With Assumption 3.7a), all effects of the damper windings are neglected, yielding Ty) ; = 
Tlo = 9. Typically, the asynchronous torque of the damper windings is approximated 
by an additional damping term +da jwm,; in (3.16), which is proportional to the deviation 


Wm,i = Qm,i — Or a of the mechanical angular velocity of the rotor from its nominal value 


Q}, In particular, inserting (3.14e) with Tjo, = 0 into (3.14a), and (3.14f) with Too; = 0 
into (3.14a) yields 

Uga, m Uj i Re = Rs ila (3.17a) 

Digs = U4 i + Iai X4 Balas (3.17b) 

Tio, roe = oe. = Ui + Lai . (Xai _ Xba) (3.17c) 

40, Ua, fin —U4 i T Tq, ` (Ka; =f Kal (3.17d) 

Vi = Omi, (3.17e) 

Is + Dam. + da iWm,i = Mri — Mei. (3.17£) 
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Figure 3.4: Equivalent circuit diagram of a SM in transient state connected to the grid. 


With Assumption 3.7b), it holds that Xq, = X/ Ui = 0, and R,,; = 0. This leads to 


qv 
Uga,i = Lagi X Gir (3.18a) 
Uggs = Uri + LaiXair (3.18b) 
Tho ia = Un — Uys + Tan: (Xai — X43); (3.18c) 
Yi = mi, (3.18d) 
dm, + Das Omi + da, iwm i = Mri — Mei- (3.18e) 


The dynamics of the electrical subsystem is illustrated in the left half of Fig. 3.4. 


Electromechanical Coupling 


On the mechanical side, it usually holds that Da; << da, [MBB12, p. 455], thus Dg; can be 
neglected in (3.18e). With the above assumption and OP = 0, we get wm, + dajiwmyi = 
Mr; — Mei. Due to the fact that power equals torque times angular velocity, this yields 


1 
Qui 


Jimi + da,iwm,i = (PT.i — Pei). (3.19) 


Multiplication by De and Assumption 3.6a) lead to 
OT +8, 5 > da,iWm,i = PT,i — Pe,i- (3.20) 
Now we switch from mechanical to electrical domain by choosing 


20 mi 20% 
Q = > ’ RT = > , (3.21) 


npp Mpp 


where npp € N denotes the number of pole pairs. The deviation of the electrical angular 


velocity from its nominal value is thus given by w; = Qi — Qn, = Zomi, Accordingly, we 
R PP 
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get ùi = 0; = nd cz Expressing the mechanical velocity in (3.20) by means of the 
PP 


Npp 
electrical velocity w; yields thus 


Ji- rer ; a i + dai: a eae ; a Wi = PT,i — Dei; (3.22) 
0 U 
=I; =A; 


which is the classical second-order swing equation. 


Connection with the Power Network 


The SM is connected to the power network as depicted in the right half of Fig. 3.4, where Ü; 
denotes the phasor of the ”external“ nodal voltage U; at the coupling point with the power 
network. The active and reactive power injection at the coupling point are given by”! 


Pinj,i = Xana sin(0y i) + 5 aa sin(20, 4), (3.23) 
U!U; U? 
dinji = Ks cos(dn,;) — er (Kane sin? (Vni) + Xani cos” (di) : (3.24) 


In case of non-salient pole SMs, we have Xa,i = Xq,; and thus Xan,i = Xqn,i. Therefore, the 
above equations simplify to 


WU; 
Pinj,i = 2 sin(dn,); (3.25) 
dni 
UU; U? 
taii = Tu cos(dn,;) — Kar (3.26) 


The coupling between mechanical and electric domain is given by 
Pe,i = Pinj,i; QS (3.27) 
With Qn i = Mi + wm, and %; = Omi, we get Š; = Wm,i. With Assumption 3.6a), it holds 


that npp - Š; = 6;, thus (3.18d) is equivalent to 6; = w;. This allows to write the overall 
equations compactly as 


6; = wi, (3.28a) 
Tiwi = — Aw; + PT,i — Pinj,i» (3.28b) 
ThoU} = Ura — U} + (Xa - X4) (UN dinji (3.280) 


31 A detailed derivation of this physical relationship is provided in Appendix B.1. 
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Figure 3.5: Single-phase circuit diagram of a grid-forming inverter with capacitive inertia. 


3.2.3 Inverter-Type Nodes 


The vast majority of DERs fed by RESs are connected to the grid via inverter interfaces. 
The model for Z nodes derived in the following is able to both represent the main physical 
characteristics of the inverter dynamics while also ensuring a seamless integrability into the 
existing SM infrastructure. 


At each Z node, we consider a grid-forming** inverter [RLBR12] with capacitive inertia and 
controllable DC current Ipc,;. We assume that each inverter i € Vz fulfills the following 
assumptions regarding its inner control loop: 


Assumption 3.8. a) At AC side, the inverter is able to produce a perfect sinusoidal 
AC voltage Uj. 


b) There are no switching losses in the inverter. 


c) The inner control loop of the inverter is equipped witha Upc ~ w matching controller 
[JAD 16]. 


The validity of these assumptions is ensured as follows: To comply with Assumption 3.8a), 
the AC current can be processed through an appropriate LCL low-pass filter to suppress the 
harmonics caused by switching dynamics of the involved insulated gate bipolar transistors. 
Assumption 3.8b) is justified by the fact that the losses caused by the DC-side conductance 
Gpc,i are significantly larger. The Upc ~ w matching controller that was established in 
Assumption 3.8c) allows to exploit the structural similarities between kinetic energy of the 
rotor of a SM and electric energy stored in the DC-side capacitor of the inverter. 


A key difference to alternative concepts of mimicking SM dynamics, such as the virtual 
synchronous machine concept, is that the Upc ~ w matching controller directly uses the 
physically available electric capacitance Cpc,; instead of relying on a separate storage and/or 
a microprocessor-based external emulation of SM dynamics which may be more susceptible 
to delays. By contrast, using the Upc ~ w matching controller as inner control loop for 
frequency and voltage control purposes has been explicitly promoted in the recent paper 
[DBSPG19]. 


32 Inverters are said to be grid-forming if they operate as controllable voltage source. By contrast, the inverter 


is said to be in grid-following mode, if it acts as an (active and reactive) power source while U; and 6; are the 
dependent variables [SFO15, p. 69f.]. 
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With Assumption 3.8a), the inverter can be represented by its single-phase equivalent circuit 
which is depicted in Fig. 3.5. The constitutive equation for the DC-side voltage Upc, is 


1 
Roc,i 


Cyc,iUpei = Upc — Ki + ipai (3.29) 


In grid-forming mode, the DC-side voltage Upc,; serves as independent variable which can 


be chosen freely. With Assumption 3.8c), the phase angle 0; of U; is chosen according to the 
integral controller scheme [JAD16, MDS* 18] by 


Å; = Uns. (3.30) 


with k; = Mi/Udc where Ode; denotes the nominal DC link voltage. With (3.30), the 
DC side follows the dynamics 


Coc,i 04 1 
ki kiRpc,i 


Q; = Lyi + Ipc,i- (3.31) 


With Assumption 3.8b), it holds that I, ¿Ux ; = ,iUpc,i = PDC, = Pinj,i- Furthermore, we 
choose 


Qn i Pg,i 
Ipc = ——— + 3.32 
DC,i Roc. . kj Q ’ ( ) 
where pg i is a desired active power input or output. Since Q; = cy, (3.31) reads as 
CDCi . 1 1 
k Wi ṣ P w=; (Pg,i — Pinj,i) - (3.33) 
Multiplication with ©, ; along with Assumption 3.6a) leads to 
Coo,i(UBo,i)* i (Use) See ang (3.34) 
Ox i Oni Rae. i = Ps,i — Pinj,i> : 
u A 
=Tz,i =:Az,i 
(u*. .)2 x» 2 
where we can define Iz; := ene eee as the virtual inertia and Ar; := re as the 
virtual damping coefficient. In summary, this yields the inverter AC side dynamics 
6; = wi, (3.35a) 
Dz jw; = —Az ,iwi + Pei — Pinj,i- (3.35b) 


In particular, we clearly see the structural equivalence between (3.35) and the swing dynamics 
(3.28). However, both the resulting (virtual) inertia 'z,; and (virtual) damping Az,; are 
typically much smaller in magnitude than I’; and Aj, respectively (see [JAD16, Remark 2] 
and [MDS* 18, Remark 2]). As a result, a higher damping would necessitate a larger parallel 
conductance Gpc,; and thus provoke a higher power loss on the DC side. Likewise, a larger 
virtual inertia [z ; requires a larger capacitance Cpc,; and/or a larger nominal DC voltage 


* 
Une; 


3.2 Modeling of the Physical Network 43 


3.2.4 Load-Type Nodes 


Load-Type nodes represent the power supply or demand of retail producers or consumers. 
These network participants are connected via step-down transformers or grid-feeding inver- 
ters, which are each modeled by a constant impedance Ye i = ge; +7be,; plus an uncontrollable 
active and reactive power demand” along with a frequency-dependent part with an unknown 
damping coefficient A; ; > 0. Accordingly, we get 


6; = wi, (3.36) 
Pinj,i = —9eiU? — Acai — Pos, (3.37) 
Ginj,i = —be iU? — ei: (3.38) 


Remark 3.9. Distributor nodes with pe i = qe, = Aci = 0 can be eliminated from the model 
by Kron reduction [CT14; SFO15, p. 45ff.]. © 


Despite the sign convention for £ nodes, pe; and q¢,; may be negative as well. 


3.2.5 Overall Model 


In the following, the dynamic equations of S, Z, and £ nodes are combined to a novel, network- 
preserving overall model. For all S and £ nodes, we choose p; = Pinj,i and qi = qinj,i- Z nodes 
are connected via p; = Pinj,; and Uac, = U;. Moreover, we assume that all shunt admittances 
Ysh,i Of L nodes are incorporated in the diagonal elements of the nodal admittance matrix 


Ý, 


Now we define the extended physical graph 9 = (V, £) with V = V, U Vo and € = Ep U Ef, 
where Vs represents the SMs’ internal voltages Uj, and E,, represents the SMs’ connections 
between internal voltage U/ and external voltage U;. For convenience of notation, Xan, is 
included in the line impedance. Then, we can define the extended nodal admittance matrix 
Y=G+ )B incorporating both the admittances of power lines and the reactances of SMs. 


33 According to the node type, we use the passive sign convention. 
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This finally leads to the compact notation 


Dij = Wi — Wj, GEE, (3.39a) 
Li = —Aju + pg — pes — U? Gu 
— So U,U; (Bi; sin(d;;) + Gij cos(d,,)), i € Vz,  (3.39b) 
JEN; 
Tio Ui = Uta — Ui — (Xai — X44) 
( = U;, Bi + 5 Uj (Gij sin(d;;) < Bij cos(d;;)) ): LE Vs, (3.39c) 
JEeNi 
0 = - Aw; — pei — U? Gu 
- So UU; (Biz sin(¥iz) + Gij cos(d,;)), i€ Ve, (3.39d) 
JEN; 
0 = =q; + UP Bir — ), UU; (Gij sin(¥ij) — Bij cos(Vig)), € Ve, (3.39e) 
JENi 


where Vg = Vs U Vz, Ve = Ve U Vs, Li = Liwi, Gij = R{[Y]i5}, Bij = S{[V ij}, and 
N; denoting the set of neighboring nodes j € V of the extended graph Z without regard to 
edge directions. 


By defining the plant state vector £p = col{J, Ls, Lz, U s, we, Ue} of (3.39), where for all 
MEE, J EVE, k E€ Vr, lE Ve: 


V := Colm {0m}, Ls :=col,{L;}, (3.40) 
Lz := col, {Lg}, Us := col, {U;}, (3.41) 
Weis col; {ur}, U, = col, {U;}, (3.42) 


we can set up the plant Hamiltonian as 


1 2 
B Yai (3.43) 


to describe the total energy stored in the system. The first row of (3.43) represents the shifted 
kinetic energy of the rotors and the magnetic energy of the SM circuits, the second row 
represents the “virtual” kinetic energy at inverter-type nodes, the third row represents the 
magnetic energy of transmission lines and the fourth row represents the local deviations of 
load nodes from nominal frequency. 
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Using the plant Hamiltonian (3.43) along with its co-state zp = VH(z,)andV; = Vs U Vz, 
with 


OH, ` . . 

39,, 7 Put d; sind), (i, 9) €E, (3.44) 
ij 

OH, | 

DL; = Li/T; = Wi, ve Ve; (3.45) 

OH, U; i 

i a JEN; 

OH, | 

au. T > BuU; cos(vi;), i€ Ve, (3.47) 
ý JEN: 

H, 
. TWh, i € Ve, (3.48) 
Ow; 


we can set up a port-Hamiltonian representation for the novel physical plant model (3.39) 
as 


0 | 0 De 0 Dy, 0| [0 0 0 00 ] 
L -Dgs 0 0 0 0 0 A, 0 0 0 
Us| = 0 0 0 o 0 -|0 0 Rs 0 O0||z, 
0 -Dx 0 0 0 0 000 A 0 
0 0 0 0 0 0 0 0 0 0 JU, 
— e a u ee 
Ep Jp Rp 
0 0 0 0 0 
De I 0 0 -t, s 
- los|+| 0 îy 0 0 E (3.49) 
Pe 000 -L K 
[27 0 0 -I O 
Sa 
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where 
L = col{Lg, Lz}, (3.50) 
A, = diag, {Ai}, i E€ Ve, (3.51) 
A; = diag,{A;}, i E Ve, (3.52) 
Rs = diag; { (Xai - X) /TUi}, i € Vs, (3.53) 
Ü, = diag;{U;}, i € Ve, (3.54) 
Pg = col;i{Gu U? + X. GigUiU; cos(d;5)}, i € Vg, (3.55) 
JEN: 
Pe = coli {Gu U? + X` GigUiU; cos(d;;)}, i € Ve, (3.56) 
JEN: 
05 = col;i{ Rg XC CyV;U; sin(vi)}, i EVs, (3.57) 
JEN: 
a = coli{ X` GyU:U; sin(d;;)}, i € Ve, (3.58) 
JEN: 
Ty = diag;{1/Taoi}, i€ Vs, (3.59) 
Te = [Ingxng Ongxne] , (3.60) 
T= One. Inen]. (3.61) 


The input vector in (3.49) is composed of the control input up = col{p,, Ur} and the distur- 
bance input d = collg,, pz}. 


Remark 3.10. Note that (3.49) is a nonlinear system, whereby the main nonlinearities appear 
in the co-state equations (3.44), (3.46), and (3.47), caused by nonlinear electromagnetic relations. 
Moreover, there is a modulation of the dissipation matrix R, with the state vector Ue. The 
interconnection matrix Jp, however, contains only entries which are constant in time. © 


Remark 3.11. Even in the case that certain parts of Assumption 3.7 do no longer apply, the 
port-Hamiltonian framework can principially be applied to formulate a corresponding (higher- 
order) system model for S nodes. In [SDv16], a port-Hamiltonian model for multi-machine power 
networks composed of sixth-order SM models (3.14) is established. A systematic derivation of 
port-Hamiltonian representations for all classical SG models based on first-principles modeling is 
conducted in [SDv19]. © 


3.3 Modeling of Participants’ Interaction 


This section presents a model for the network participants’ interaction and competition on 
the energy market. The conceptual classification into WPs and RPs is based on the premise 
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that WPs can shift their own production or consumption strategically to some extent** based 
on a profit-maximizing strategy. Thus, WPs are competing players on a cross-zone electricity 
market which pursue to maximize their own profits P, ; through (positive or negative) power 
injections pg ;, while incurring power production costs Cg i. 


Assumption 3.12. The cost functions Cg ;(pg,;) are strictly convex. 


Remark 3.13. Assumption 3.12 is typically mild for both consumers and producers [BV00, ZP15]. 
This is due to the fact that WPs tend to internally work with a strictly convex surrogate function 
in order to apply their pursuit of profit maximization and to avoid being stuck in sub-optimal 
operating points (see the discussion in Appendix B.3) © 


By contrast, RPs typically perform their decision-making concerning power production or 
consumption independently of other RPs or WPs and have no means to displace their instan- 
taneous (positive or negative) demand. Thus, RPs are assumed to be inelastic. 


Remark 3.14. The role of WPs is congruent with the definition of CVPPs (cf. Subsection 2.2.1), 
although the formalism presented in the following also allows negative power injections. Moreover, 
the duties and responsibilities of CCs as well as their geographic coverage strongly resemble those 
of a TVPP. However, the latter terminology is avoided in the context of this dissertation, since 
the attribute “virtual” negates the active role of CCs in the mutual coordination of supply and 
demand under consideration of physical network conditions. Furthermore, our concept of WPs 
does not require unidirectional power flows as might be suggested by the term “power plant”. ©} 


We summarize the number of WP nodes as ng = ns + nz. WPs inject electrical power 
Pg € IR” into the power system and are able to influence the nodal voltages U € R3. Both 


S and 7 nodes have to meet upper and lower limits on power generation p, ; and p, ,. Note 


that all buses 2 € V may be equipped with RPs and thus with an uncontrollable active and 
reactive power demand pg ; and qj. 


For compactness of notation, we introduce cell-aggregated variables for the power gene- 
ration P, = colkez {Pen} € R”? where Psk = eyes „Posi the power consumption 


Po = Colpez {pen} € R”= where pek = Vievs p Plis and the active power loss ĝ = 


colkez {Px} € R”z, where , = Vievs k Dj. 


The following subsections pave the way towards a market-based control scheme with zonal (i.e. 
spatially differentiated) prices A, € IR". Subsection 3.3.1 derives an incentive-compatible 
market mechanism which respects the requirement of unbundling. Subsections 3.3.2-3.3.4 
model the behavior of WPs, RPs, and CCs, respectively, under the chosen market mechanism. 
Finally, Subsection 3.3.5 summarizes the consequences of market interactions on the upcoming 
controller design. 


34 Hence, spatially differentiated prices also provide medium- and long-term incentives for shifting capacity to 


other cells that, on average, provide higher prices for generators or lower cost fro consumers. 
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3.3.1 Design of Market Mechanism 


In the classical model of energy economics, a market equilibrium of the competitive power 
system from Definition 3.3 is attained if the market price A resulting from competition yields 
a balance between total demand 1! p, and total generation 1'p, (see the common Definition 
A.23 in Appendix A.6). However, this definition makes no statement whether the achieved 
market equilibrium is “valuable” from the invididual participants’ point of view or from a 
global perspective. The individual utility of an equilibrium is often certified by means of the 
Pareto efficiency (or Pareto optimality), cf. Definition A.5. According to the first theorem of 
welfare economics (cf. Theorem A.25), each market equilibrium is also Pareto efficient if the 
participants compete over a perfect market. This prototypical notion of an idealized market 
contains numerous simplifying assumptions and is summarized in Definition A.24. 


For the normative evaluation of the global efficiency of an attained equilibrium solution, there 
exist different metrics such as the utilitarian welfare function” or the Rawlsian welfare function 
(cf. Appendix A.6). When adopting the utilitarian welfare function as a measure for global 
efficiency, the market equilibrium achieved in a perfect market is not only individually, but 
also globally efficient (cf. Remark A.27). It can thus be stated that such a perfect market should 
always be targeted by means of any appropriate market design. However, certain properties 
of real-world power systems are diametrically opposed to those required in Definition A.24: 


(1) The economic model of market interaction is based upon the assumption that both 
suppliers and demanders are price-elastic. However, our above model resulting from 
Definition 3.3 also incorporates inelastic participants. In particular, we have adopted 
the natural assumption that the supply or demand of inelastic RPs triggers the supply of 
elastic WPs. 


(2) In real-world electricity markets, supply and demand is settled over an interconnected, 
dynamic power system (3.49), which may be subject to a number of additional opera- 
tional constraints, e.g. due to transmission capacity limits or security constraints to 
prevent voltage collapse. By contrast, the plain economic model does not take into 
account any underlying physical structure (see the copper plate assumption discussed 
in Section 2.1.2). 


(3) The plain economic model does not reflect any transmission losses that always occur in 
real-world power systems. 


These points emphasize that, when dealing with actual electricity markets under realistic 
operating conditions, a mere reference to the existing theorems of welfare economics is 
insufficient. In the subsequent sections each of the above challenges is therefore intensively 
studied when deducing a viable (i.e. network-preserving) market-based control scheme. 


In the remainder of this subsection, we first address the issue of inelastic market participants 
(challenge (1) above). As outlined in Subsection 2.2.2, the two main philosophies for modeling 
distributed market-based interaction of noncooperative agents are Bertrand and Cournot 
competition (see Subsection 2.2.2 for a discussion of current research approaches related to 


35 The maximum of the utilitarian welfare function is called social optimum and is characterized by equal marginal 


profits of each participant. 
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electricity markets and [dD21, Chapter 1] for a more general economic framing). We thus 
investigate whether these two models of competition are able to generate (individually and/or 
globally) efficient market equilibria in case of inflexible RPs. For didactical reasons (yet without 
loss of generality), we exemplify these considerations on an atomistic model by requiring 
Pei > Oand pe; > 0, and by temporarily neglecting both the system dynamics (challenge (2)) 
and the line losses (challenge (3)). Both challenges are then considered again from Subsection 
3.3.2 onwards. 


Bertrand Competition with Inelastic Demand 


In Bertrand competition each WP i € W bids a price A; € R per amount of power to maximize 
its own profit 


PA Ami) FA i (Ai, A~) AR), (3.62) 


where Cgi : R — R>o represents the costs of WP i to generate pg ;. The power generation 
invoked by the system operator is given by 


i 0, Jj € W\{i} Ay < Ai, 


Dg ilAi Ai) = 4 aT, ; (3.63) 


N°? 


otherwise, 


where N is the number of WPs that bid the lowest price A = min;{X; }. By (3.63), WP i is not 
able to sell any power if there exist other WPs that bid a lower price. Likewise, if WP i bids 
the lowest price A; = A, then the total demand 11' p; is equally shared among all WPs that 
bid the lowest price A. 


Bertrand competition has many undesirable properties. In [Das95], it is shown that even 
for strictly convex cost functions Cg ;(pg,i) (cf. Assumption 3.12), Bertrand competition may 
exhibit non-unique market equilibria. In [Wei06], it is elaborated that the range of acceptable 
prices is determined by A; = [A?, A?], where the outside option price 


1'p 
X= Cs,i ( 2 


(3 Ip, 
N 


(3.64) 


is the price below which player i earns a negative profit when sharing the market, and the 
competitive price limit A? defined by 


1'p . 5 
Poi ( K a) =P,,(1'2, AP). (3.65) 


Bids A; > A? would always attract the other players to bid A~; = X; — €, € > 0, such that 
they share the total market exclusively among each other. Hence, the intersection of all those 
intervals constitutes the set of all possible Nash equilibria (cf. [Wei06]) 

Lnash = TA : Anax = A < Main }> (3.66) 


e 
min 


where Af ax = maxjep{A?} and Afin = min;ep{A?}. This is demonstrated by the following 


two-player example. 
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Example 1 


Consider two WPs competing in a Bertrand competition, whose individual cost functions are 
given by Cg.1(Pg,1) = 4p2 1 — Pg,1 and Cg, 2(Pg,2) = 4p? a. This results in the profit functions 


Pei (Ai, A2) = À1Pg,1 (A1, A2) — Ap? (Àr, A2) + Pg, 1 (A1, Aa), (3.67) 
Pg,2(A2, A1) = A2Pg.2(A2, Aı) — 4p2 9(A2, A1). (3.68) 


Let the total demand be 1'p, = 1, then (3.63) takes the form 


1, Ai < Aa, 1, Aa <Aı, 
Pe (Ar, Aa) = 0.5, Ai = Aa, 5 Ds.2(A2, Aı) = 0.5, Aa = Àl, (3.69) 
0, Ay > As, 0, Ag > 1, 


and inserting (3.69) into (3.68) yields 


ài- 3, Ay < Aa, Ag — 4, Aa < Ai, 
Pei (Ai, A2) = $ 0.5A, — 0.5, Aı=A2,, Pea(A2,A1) = $ 0.5A2g -1, Ao = Aq, 
0, Aı > ro, 0, Aa > A. 

(3.70) 


The profit functions (3.70) are depicted in Fig. 3.6. 


Figure 3.6: Exemplary profit functions for WPs 1 (blue) and 2 (red). 


The blue lines correspond to WP 1 and the red lines correspond to WP 2. The solid lines show 
the WPs’ profit if they were to capture the full market (pg; = 1), while the dashed lines 
show their profit if they were to share the market (pg i = 0.5). Possible market equilibria can 
now be determined by construction: From Fig. 3.6 it can be seen that WP 2 will never choose 
Ag < 2 as this would not result in a positive profit. Moreover, since for all Ay € [2,5] it is 
more profitable for WP 2 to share the market than to capture the full market or to capture 
nothing, the best option for WP 2 is to choose Az = Aı. The same reasoning holds if WP 1 
were to choose its price 1 freely given Aa € [2,5]. Consequently, all A € [2,5] are market 
equilibria. By contrast, if WP 2 were to choose Aa > 5, WP 1 could make a higher profit by 
choosing Aı < Aa and capturing the full market, which is why A > 5 cannot be a market 
equilibrium. The outside option prices (zeros of the dashed lines in Fig. 3.6) are A] = 1 and 
à = 2, and the competitive price limits (intersection of dashed and solid lines) are A} = 5 
and à$ = 6. The intersection of the intervals [A$ , \$] and [X3, A$] is thus equal to [2, 5]. 
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Remark 3.15. Ifall players are aware of each other (i.e., know the number ofall competitors as 
well as well as their individual cost functions), it is the best choice for them to choose A; = A in- 
If the players are not aware of each other, then any equilibrium A; = [AS * in| is stable. This 


can be justified as follows: Let A? € [AS ax, A$ in] be some equilibrium. If player i performs a 


min 
so-called e-undercutting [Cab17, p. 105] by choosing XS = aË — € to test if it can share the 
total market, this will result in lower profits PAY Ab) < Pg i (Aĵ, ARSE, thus no player has 


an incentive to refrain from \; = BY and hence X* is a Nash equilibrium. © 


The fact that there exists no unique market equilibrium is undesirable, as it causes the actions 
of market participants to be unpredictable and path-dependent. Consequently, it becomes 
difficult to incorporate tangible higher-level objectives into the market mechanism. Another 
issue with Bertrand competition is that the resulting equilibria may not be (individually) 
economically optimal. We show this by means of a second example. 


Example 2 


Consider two WPs with cost functions 


1 
Cg,1(Pg,1) = Pers Cg,2(Pg,2) = Pe»; (3.71) 


3 
which compete over a total demand 1'p, = 10. Following the same procedure as in Ex- 
ample 1 above, it can be shown that Ai = Aa = 5 is the unique market equilibrium. Both 
WPs generate p3 ı = Pz. = 5 and their corresponding profits are Pg 1 (Af, A3) = 0 and 
Po 9(A3, AT) = 16.33. However, there exists an infinite amount of alternative equilibria 
which are Pareto dominating (both in terms of higher profits and lower costs for WPs). For 
example, if the generation were distributed p} 5 = 2.5 and pi» = 7.5, the individual profits 


were PË (At, A$) = 6.25 and PË (Af, A$) = 18.75. 


Through the equal allocation of power generation?” as in (3.63), Bertrand competition may 
cause large market barriers: If a new WP enters the market with ny WPs already competing 
in a Bertrand competition with equilibrium price *, this new WP now has to bid * and to 
generate 1'p,/(N + 1) (which might be very large**), in order to still obtain positive profits. 
This requires WPs which want to enter the market to make large investments beforehand and 
therefore prevents many small-scale WPs such as RESs to participate in the energy market, 
thereby creating wrong incentives that are considered anti-competitive. 


36 
37 


since, figuratively speaking, the solid profit curve is below the dashed profit curve (cf. Fig. 3.6). 

Some recent approaches, e.g. [CC20], overcome the problems of a pure Bertrand competition by introducing 
a second step after the price bidding where the power generation p, is divided unequally according to the 
individual preferences of WPs. In particular, WPs disclose how much power they would prefer to generate given 
the equilibrium price A. Afterwards, the total demand 1 p, is then distributed accordingly. This modification 
however has the main disadvantage that WPs have to disclose their preferences towards a centralized system 
operator, which is a contradiction to the overarching concept of a competitive power system with cell-based 
infrastructure which was adopted in Definition 3.3. 

If there is no WP that can meet the total demand at its marginal costs, it can be shown that there exists no 
equilibrium (Edgeworth paradox, cf. [KO93, p. 328)). 


38 
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Cournot Competition with Inelastic Demand 


In Cournot competition, each WP i € W sets its power generation pz; to maximize its own 
profit 


Pg i(Pg.is Pg,-i) = A(1'P,)Pe.i — Cg,i(Dg,i)- (3.72) 


In contrast to Bertrand competition, where the power generation of each WP i € W is a 
function of the price bids A, in Cournot competition the unique market price is set by the 
system operator as a function of the total power generation, while each WP chooses its own 
power generation pg ; by itself. 


For the classic Cournot competition, it is required that all consumers are price-elastic in order 
to arrive at a meaningful market equilibrium and deduce the price function A(1'p,) (see 
e.g. [DM19]). However, as discussed at the beginning of Section 3.3, we choose to model 
consumers as price-inelastic. In a classic Cournot competition, this would trigger all WPs to 
tactically choose their p, ; such that 1'p, < 1"p, in order to maximize profits. As a result, 
the prices would eventually grow infinitely large (yielding infinite profits for all WPs) and 
thus no market equilibrium could be reached at all. However, in order to still achieve a market 
equilibrium in a Cournot competition with price-inelastic consumers, we make the following 
assumption: 


Assumption 3.16. WPs do not know the total demand 1.' p, nor how the market price X 


depends on the power injections p,. 


Assumption 3.16 is met if strict observance of information barriers between WPs and CCs can 
be ensured (cf. [LPWL07, p. 24ff.]). As a result of Assumption 3.16, WPs act as pure price-takers 
as they are unaware of their own influence on the price X. Note that it is not uncommon for 
WPs to be forced to act as price-takers in order to keep competition equitable, especially on 
critical services like power generation (cf. e.g. [Con14, p. 82]). 


Owing to a price-taking behavior, each WP i € W now solves 


max Pg i(Pg,i) = APg,i — Cg,i(Pg,i) (3.73) 
ki 


in order to maximize its profit given the price A, which is the solution of 
1'p, (A) = 1" pp. (3.74) 


As soon as the exact function A(1'p,) is not known, Cournot competition has some very 
appealing features: First of all, the resulting market equilibrium is unique. Consider WPs with 
profit functions as in (3.73). Since the sensitivity of pg; on A is not known, WPs will always 
produce at marginal cost such that VC, ;(pg,;) = A. With Assumption 3.12, VC, ;(Pg,;) is 
strictly monotonically increasing and thus p4 ;(A) = (VCg,i)~*(A). Under the condition of 
supply-demand matching, we therefore have J),.p(VC,,;) (A) = 1" py, which is again 
strictly monotonically increasing, thereby leading to a unique solution A. More general 
statements on uniqueness of Cournot equilibria are provided e.g. by [MQ18, Theorem 4.1]. 
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Example 2 (cont’d) 

Recall Example 2, where two WPs with cost functions (3.71) compete over a total demand 
1'p, = 10. With Assumption 3.16, the market price X is calculated as follows: For a fixed A, 
the profit functions of WPs 1 and 2 with cost functions (3.71) are 


Pg,1(Pg,1) = Apg,ı — Peas Pg,2(Pg,2) = APg.2 — T (3.75) 
With (3.75), the optimal power generations in terms of (3.73) are 
Poi = w Pe. = x (3.76) 
Inserting (3.76) into (3.74) with 1'p, = 10 yields 
1 3 
5° + 3 =10 => M = 5. (3.77) 


Although A* is the same as with Bertrand competition (cf. page 51), the resulting power 
generations are now given by p3 ı = 2.5 and p% o = 7.5, which are Pareto dominating among 
all market-clearing solutions fulfilling (3.74). 


Despite the fact that both classical Bertrand and Cournot competition are unsuitable as soon as 
some fraction of the prosumers is inelastic, the above example suggests that using information 
barriers (in terms of unknown price sensitivities) to a Cournot model of competition result in 
the following benefits: 


1. The resulting equilibrium is stable, since WPs have no incentive to supply more or less 
than the profit-maximizing input which is located at marginal cost (cf. [NL85]). 


2. The resulting equilibrium is unique and causes minimal overall costs (cf. [MQ18, Theo- 
rem 4.1; Var10, p. 636ff.]). 


Note that both findings will be formalized to the case of general lossy networks with additional 
operational contraints later in Sections 3.4 and 3.5, respectively. 


As will be elaborated in the remainder of this subsection, those information barriers are not 
only adequate to allow Cournot competition with inelastic demand, but also to approximately 
satisfy the conditions of a perfect market (cf. Definition A.24). 


Measures to Achieve a Perfect Market under Cournot Competition with Inelastic 
Demand 


Definiton A.24 is based upon idealized assumptions that can never be fully satisfied in reality. 
For instance, each real-world WP always has a certain (i.e. finitely small) influence on the 
overall market, and both information and power flows can never propagate at infinitely high 
speed through the network. However, through a reasonable design of market mechanisms 
and interactions, the characteristics of perfect competition can be pursued to a large extent: 
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Conditions 1) and 2) in Definition A.24 are trivially fulfilled. To satisfy condition 3), it has to 
be ensured that price formation over the network occurs sufficiently fast so that each market 
participant is equipped with the same information. To meet condition 4), we have to assume 
that the portions of electrical power which can be offered by suppliers are not quantized. 
From condition 5) it can be concluded that the resulting market mechanism must not incure 
any costs in total that are charged to the participants, i.e., the net profit of all CCs has to be 
zero. To comply with condition 6), WPs should at least act as fast as possible, such that no 
internal technical limits, e.g. start-up times, turbine dynamics, or ramp-up constraints are 
violated. Finally, condition 7) has the following implication: 


Assumption 3.17. No network participant is large enough to exercise market power. 


Remark 3.18. Assumption 3.17 is justified if the overall number of WPs is high, since a 
high percentage of inelastic demand is always accompanied by an increased tendency to form 
monopolies and thus consequently higher prices [RT06]. Another means to satisfy Assumption 
3.17 is the requirement that no WP should have a substantial market share in a specific cell” 
[Sta19, SHH19]. Alternatively, if there is a small number of market participants in a specific 
cell or throughout the network, these participants must not be aware that they have a relevant 
influence on the prices. Another implication of condition 7) is that any collusion among WPs has 
to be prevented [Küh01]. © 


Remark 3.19. With the increasing penetration by intermittent RESs, the price dynamics becomes 
unpredictable [RN20]. Thus, it is increasingly cumbersome for WPs to correctly predict the future 
price dynamics as well as the price sensitivities needed in (3.72). This warrants, to a greater extent, 
the assumption that the momentary supply of WPs is at marginal cost, even if no WP has zero 
market power under real-world conditions. © 


Remark 3.20. While Assumption 3.17 allows no oligopoly among WPs, there is no such 
requirement for the overall amount of CCs as they are assumed to be non-strategic network 
participants acting as the network’s neutral market facilitators [dQQA19, p. 16]. See also Section 
7.2 and references therein for conclusive statements on optimal cell partitioning. © 


Fig. 3.7 gives a summarizing overview of the obtained key stakeholders and their physical, 
informational, and monetary interaction. Compared to the status quo shown in Fig. 2.1, there 
is a drastic reduction in communication complexity along with rigorous information barriers 
between CCs as sovereigns over the network (complete information about their own network, 
but no means to physically interact with it) and WPs as sovereigns over generation capacity 
(able to physically interact with the network, but no knowledge about the network state). 
Moreover, it is most notable that any communication from CCs to WPs is exclusively via 


3 The establishment of distributed CVPPs is thus not only welcome in terms of portfolio diversification for the 


individual WPs, but also from an overall welfare point of view to prevent local monopoly formation. 
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Figure 3.7: Schematic diagram of key stakeholders arising from the proposed control scheme. The yellow box 
encompasses those components that are elaborated in detail within the scope of this dissertation. The 
arrows mark the typical flow direction of physical power (solid lines), information (dashed lines), or 
capital (triangle lines). 


(wholesale) prices, Consequently, a full separation between the market-driven world (upper 
pane in Fig. 3.7) and the non-market-driven world (middle and lower pane in Fig. 3.7) is 
reestablished. 


In the next Subsections 3.3.2-3.3.4, detailed and appropriate cost and communication structures 
are derived for WPs, RPs, and CCs, respectively, such that both requirements for perfect 
competition (cf. Definition A.24) as well as challenges (1)-(3) from Subsection 3.3.1 are 
properly addressed. 


3.3.2 Wholesale Prosumers 


WPs are flexibility providers [AP21], where the flexibility currently provided to the network 
corresponds to the momentary upper and lower power limits p, ; and Poi Throughout the 
following considerations, we suppose that all WPs behave rationally on an individual basis 
(cf. Assumption A.4) by means of a pure profit maximization strategy. To comply with the 
unbundling paradigm, we require the following: 


Claim 3.21. There should be no need for WPs to disclose their own cost or profit function 


to any opponent or CC. 
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From Assumption 3.17 and Claim 3.21 it immediately follows that each WP 7 € W neither 
has knowledge about the individual components nor about its own influence and thus seeks 
to maximize the momentary profits, since there is no personal benefit in pursing a dumping 
or markup strategy. Another beneficial implication of Claim 3.21 is that there is an intrinsic 
motivation for WPs to minimize their own cost structure. Namely, a flattening of C, ; leads to 
WP 7x being allocated more shares of total demand due to the merit order effect [ZPE17, p. 279£.]. 
Consequently, every cost reduction of WP m € W also increases the own revenue Ry (while 
reducing the revenues of competitors). 


To prevent any additional side payments, we further assume that all external costs are in- 
ternalized appropriately into the cost functions C, ;, e.g. that the individual greenhouse gas 
emissions are settled by a suitable emissions trading scheme (ETS) such as EU-ETS from 
the European Commission [ZW10]. These internalized costs can then be represented by an 
additional linear-affine component in the cost function. Typically it has a positive gradient for 
fossil-fueled CPPs that have to purchase allowances and a zero or negative slope for power 
generators from RESs that can sell their surplus allowances on the market“. Thus, typically, 
the actual costs for CPPs are higher than those of RESs. Note that these additional internalized 
costs do not affect the strict convexity assumption (cf. Assumption 3.12). 


CCs purchase power from each WP of the own cell by paying the zonal price A; per unit of 
electrical power. To incorporate the marginal contribution of WPs on frequency stability, we 
propose the following remuneration scheme for WP 7 € W: 


Rr= >) Ai Pei — Wi Pei (3.78) 
t€Vw x 


where X; = Ar, fori € Vz p. The revenue R, consists of the frequency-independent payment 
for active power production at price A; and a frequency-dependent part. The latter serves as 
an instantaneous penalization for active power generation during an overfrequency period 
and as an additional reward for active power generation during an underfrequency period. 
Subtracting the node-individual costs Cg į incurred for power generation from (3.78), we get 
the overall profit 


Pr = 2 Psi = x; —Cg,i(Dgi) + Ai Pg,i — Wi * Pg,i (3.79) 
1€ Vw, dEVw,r 
for WP m € W. In addition to maximizing P,, WPs must respect their internal operating 
limits 
Ps < Pg,i < Deis (3.80) 
U;<U;< U; (3.81) 


at each node i € V (cf. challenge (2) in Subsection 3.3.1). Constraints (3.80) represent the 
remaining power generation capacity of WPs that is not contracted on the WEM (cf. Fig. 3.7), 


40 EU-ETS is based on the “cap and trade” principle, i.e., the quantity of emission allowances is capped and will be 


reduced over time. If a certain company cuts its emissions, it can sell its remaining emission certificates on the 
secondary market. 
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whereas constraints (3.81) are typically imposed by the grid code and monitored by the local 
cc. 


Remark 3.22. For the sake of simplicity we assume that the individual cost functions Cg i 
and thus the profits P„ are not dependent on the own voltage magnitudes U;. This is a common 
assumption for both CPPs and inverter-interfaced DERs, since changes applied to their output 
voltage usually do not incur additional cost [BV00, p. 405]. © 


Remark 3.23. Despite the fact that the above expressions feature a producer-centric notation, 
they also allow to incorporate elastic consumers at node i which aim to maximize their profits 


Pei = Ui (pei) — Ai Pei + wi Pei (3.82) 


with U;(pei) being a strictly concave utility function. These consumers can be modeled in a 
straightforward manner as producers with negative generation Pg i = —pe,, < 0, since the profit 


maximization problem building upon (3.82) is structurally identical to (3.79). © 


With Assumption 3.17, Claim 3.21, and Assumption 3.12, each WP 7 € W thus aims at solving 
the constrained optimization problem 


max Pr (3.83a) 
Pg,n:Ut,n,UL,r 

subject to Di; < Pg,i S Deir TE Vie, (3.83b) 

U,<U,<U;, i€ Vwo- (3.83c) 


Remark 3.24. The reaction time of a particular WP to output a certain power p, ; or voltage 
U, varies in a wide range, depending on the specific electromechanical or electrochemical energy 
conversion technology used. For example, modern grid-forming inverters can provide the required 
quantities within less than one second [PEH18, p. 3662], while the turbogenerator of a classical 
large-scale thermal power plant may take several minutes until reaching maximum power 
[KRK* 18]. We will account for such dynamic constraints directly in the WPs’ controller equations 
to be derived in Section 3.4, so they need not be considered in the formulation of the optimization 


problem. © 


Lemma 3.25. For each equilibrium of (3.49), optimization problem (3.83) is equivalent to 


max P- (3.84a) 
Pen U£, n UT, x 

subject to De < Pg,i S Deis TE Vey (3.84b) 

U;,<U<Ui, deVwanvz, (3.84c) 


Ww, < Uti < Wi, i E Vw, N Vs, (3.84d) 
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where 


Proof. Let [)* denote an equilibrium of (3.49) and let 1 € Vs. From (3.28c) it follows that 


UF, = (U;)* — (Xa — X40) Gajya/(Ui)* =: Vi(Uj, Ui). (3.86) 


Inserting (3.26) in (3.86) and comparison with (3.85) yields UV; = W;(U/,U,) and Y; = 


Ww, (Uj, U;). With 


u a d, (coston) = ) >0 (3.87) 


under normal operating conditions, it holds that Ų; is strictly monotonically increasing with 
respect to U;, thus Us < U; = > WU; < UW; and U; < UX => Y; < W;. Accordingly, each 
optimizer of (3.84) is an optimizer of (3.83), and vice versa. 


3.3.3 Retail Prosumers 


RPs at node i € V are characterized by an inelastic active and reactive power demand pe ; and 
qe, i- The cumulative cost for consumption in cell k € Z are set to 


Ch = Ape: 5 Pei (3.88) 


iEVZ,k 


where Ag ;, denotes the retail price for electricity in cell k (cf. Fig. 3.7). 


Note that small-scale power plants relying on uncontrollable power sources (e.g. single 
photovoltaic power stations or wind power plants that are not integrated into a VPP) can 
be modeled as RPs with pe; < 0. In addition, pe; and qe; are used to cover all positive or 
negative physical supplies arising from other contracts such as forward contracts or obligations 
due to the day-ahead market. However, from these markets only the current value of the 
resulting market clearing quantities is known to the local CC, while none of the other network 
participants are aware of it. Furthermore, it is completely unknown for all network participants 
which active and reactive power flows are to be expected in the future. 


3.3.4 Cell Coordinators 


The CCs’ exclusive responsibility for stable, economically efficient and grid-supportive opera- 
tion which was outlined in Section 3.1 has two connotations: On one hand, exclusive means 
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that no other network participant (neither CC nor WP) shares the same information or has the 
same responsibilities. On the other hand, it means that maintaining the above three objectives 
is the sole purpose of each CC. In particular, it does not have any economic goals. As pure 
service providers, CCs solely own a portion of the transmission and distribution infrastructure 
(cf. challenge (3) in Subsection 3.3.1) and, apart from that, neither storages nor any kind of 
generation capacity. This is a direct consequence from the unbundling requirement elaborated 
in Section 2.1. The geographic region that is covered by CC k € Z is characterized by the 
weakly connected component % p in G. 


Assumption 3.26. CCs have full knowledge about their cell’s own line parameters (Gij, 
Bij, gi, bi) and maintain continuous, delay-free measurements of the voltage magnitudes 


U;, the power injections pg i, the power consumptions pe i, qe i, and the power flows P;j, 
Qij within their own cell. 


Assumption 3.26 can be justified as follows: The line parameters can be derived from manu- 
facturers’ specifications using (3.5) and (3.6). The power flows and voltage amplitudes can be 
measured directly in-situ by means of phasor measurement units (PMUs) [MBB12, p. 61f.]. In 
particular, note that the voltage angles 0; or ¥;; do not need to be measured. Moreover, CCs 
are only aware of the current "grid-side" power flows as well as calculations of the momentary 
resistive transmission losses 


ô, = 5 Ga + 5 GijU:U; costau) (3.89) 


iEVZ,k JENG 


within cell k. However, CCs do not know the WPs’ or RPs’ internal parameters nor any of 
the individual network participants’ profits or optimization strategies. Furthermore, if some 
of these quantities are not directly measurable, state observers can be used to reconstruct 
suitable estimators for each of the respective quantities. Such state observes are already being 
used successfully in today’s power systems. Recently, researchers have also increased their 
scrutiny of new methodological research on state observers tailored specifically for power 
systems. Comprehensive overviews of current research branches in this field are provided 
in [ARO+ 18, WGCS19, KKS20]. 


For all i, j € Vz,x, the responsible CC k € Z seeks to choose the cell-specific price Aj, = 
A; = A; in such a way that its own (virtual) profit P,, determined by the net active power 
balance within the own cell, is zero: 


Pr = Ar: @ + 5 2 — Ar: 5 Pg,i =0. (3.90) 


ieve iEVZ,k 
iVe 


3.3.5 Summary 


Using the prototypical network participants WPs, CCs, and RPs, Subsections 3.3.2-3.3.4 have 
modeled the different operating maxims of stakeholders in a fully unbundled power network. 
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Thereby, WPs m € W constitute the players of an electricity market game with the goal 
of maximizing their own profit while complying with operating constraints e.g. for their 
own power injections and voltage magnitudes. While choosing their own actions as best 
response to given environmental conditions, WPs rely on information about their own cost 
structure and machine parameters as well as the current cell-specific price Ay. In the presented 
framework, there is no hierarchy in the execution of individual actions. As a consequence of 
the requirement of information barriers and unbundling, the individual players are not aware 
of the formation of prices. Thus, the profit functions of WPs are completely decoupled. 


At the same time, CCs as pure service providers must ensure adherence with balancing 
constraint (3.90) through appropriate choice (i.e. real-time adjustment) of cell-specific prices 
Ar while considering real-time measurements of the RPs’ power consumptions as well as 
cell-specific transmission losses. 


The dynamic interplay of these different network participants with time-varying environ- 
mental conditions (induced e.g. by load jumps or changes in the network structure) leads to 
a superposition of constrained optimization problems (3.83) and (3.90). The individual con- 
straints include input constraints (e.g. limitations of power pg ; (3.83b) or inverter voltages U; 
in (3.83c)), output constraints such as limitations of voltage magnitudes at generator terminals 
in (3.83c), and dynamic system constraints which are enforced by the physical relationships of 
the interconnected plant system (3.49). 


As already shown in Lemma 3.25, output constraints (here: constraints (3.83c) on the terminal 
voltages U;) can be translated into input constraints (here: constraints on the excitation 
voltages Uf; (3.84d)) provided that a suitable steady-state map (here V;(U;) as in (3.85)) is 
known. While for input and output constraints asymptotic satisfaction is permissible, i.e., a 
temporary constraint violation is allowed in the course of solving the optimization problem, 
system constraints, on the other hand, are complex environmental conditions that typically 
have to be satisfied even during transients in order to achieve a result that is physically 
admissible at any point in time, i.e., in accordance with the laws of physics. However, due 
to the numerous incorporated information barriers, there is no central authority that has 
complete knowledge of the whole plant state xp. A suitable controller design must thus take 
these information barriers into account, but at the same time must exhibit favorable properties 
from an overall system perspective such as robustness to disturbances and sufficiently low 
computational effort of local calculations. With this in mind, the next section provides the 
main result in the sense of Contribution 1. In particular, the derivation of distributed controller 
equations enables network participants to achieve the overall goal of a stable overall power 
system by means of individual and local control actions as required by Contribution 1. 


3.4 Controller Development 


For optimization-based control on constrained physical systems, arising from the interconnec- 
tion of the physical plant system from Section 3.2 with the network participants from Section 
3.3, two different branches are known in literature. The first branch emanates from a classical 
understanding of solving optimization problems in a feedforward manner, whereby there exists 


3.4 Controller Development 61 


a preferably exact steady-state model for the plant system, allowing its direct representation 
as an additional set of constraints. MPC (see Section 2.2.2) is a classical representative of a 
feedforward optimization approach. As discussed in Section 2.2.2, there is always a trade-off 
between constraint satisfaction (i.e. real-world feasibility of the resulting OPF solution) and 
computational complexity. Thereby, feedforward optimization methods for power system 
applications typically scale poorly not only with model order and number of iteration steps, 
but also with the number of nodes per WP or CC. However, imperfectly estimated parameters, 
non-modeled side effects, or sudden structural changes not captured in the model always 
cause physically infeasible solutions at the plant input without any further possibility of 
compensation. 


As a second branch, feedback-based optimization converts the optimization algorithm into 
a continuous-time control law based on online measurements of the current plant state zp, 
while driving the physical system to its (constrained) optimum value“. By directly relying 
on measurement information, no perfect model knowledge is required. Instead, the physical 
constraints (along with possible disturbances or structural changes) manifest themselves 
directly in the measurement signal as they arise via the feedback branch, where “the physical 
system [itself] acts as a constraint enforcer” [HBHD21b, p. 2]. Since the system equations 
no longer need to be calculated separately by each local agent, this significantly reduces 
the problem size of the individual optimization problems, and thus also the computational 
effort for each network participant. Furthermore, it is no longer necessary to conservatively 
model all possible realizations of a disturbance. Instead, only the actual materialization of 
the underlying disturbance process is employed for the participants’ decision making via 
real-time measurements of the system state. 


Due to these evident advantages, feedback optimization strategies will form the methodological 
basis of all controller design in the remainder of this dissertation. Subsection 3.4.1 presents 
the main types of feedback optimization strategies known in the literature and discusses their 
advantages and disadvantages. Subsection 3.4.2 then deduces distributed feedback control laws 
for WPs and CCs. Subsection 3.4.3 analyzes stability of the closed-loop equilibrium. Finally, 
Subsection 3.4.4 illustrates the main conclusions of this section by means of an academic 
example. 


3.4.1 Introduction to Feedback Optimization 


Feedback optimization methods interpret the governing optimization algorithm as a dynamic 
control system. The basic idea of re-thinking the classical discrete-time steepest descent 


41 Although terminologically related, extremum seeking refers to optimization-based control methods for the 


model-free exploration of the feasible domain by means of approproate dither signals, see e.g. [TMMt 10] for an 
insightful literature review. However, since the necessary excitation with these dither signals is contrary to the 
requirement of economic efficiency and, moreover, an appropriate choice is highly difficult for higher-dimensional 
systems [HBHD21a, p. 6], extremum seeking-based control schemes will not be considered any further in the 
context of this dissertation. 

For quantitative robustness statements, see [CSPB19]. Practical experiments are conducted in [PBD20] and [BD19]. 
For an experimental validation of (feedforward) AC-OPF vs. feedback AC-OPF, the reader is referred to [OHC+ 20]. 
A general discussion of feedback vs. feedforward optimization of power systems is given e.g. in Section III 
of [DBSPG19]. 


42 
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approach for minimizing a certain (unconstrained) objective function J(u, y) as a time- 
continuous gradient flow was already proposed in the seminal publications [Kos56, AHU58] 
dating back to the 1950s, and has just been extensively revisited and improved’. Typically, it 
relies upon the assumption of a fast-decaying plant system whose input-output behavior can 
be represented by a steady-state input-output map y = h(u) + d. For an arbitrary objective 
function J(u, y) which depends on both the control and system output, we then get 


min J(u, y) (3.91a) 
subjectto y= h(u)+ d. (3.91b) 


Remarkably, from the controller’s point of view, constraint (3.91b) does not need to be consid- 
ered explicitly. As each output y applied to the controller already satisfies the system equation, 
it automatically holds that any input-output pair (u, y) is a solution of (3.91b). Thus, by 


defining J(u) = J(u, h(u) + d), the own control strategy can be calculated by a gradient 
descent on J. With the chain rule, we get 


u= 


asd (2) "Oh ad 
= (3.92) 


Ou Oy) ðu Mu 


It is evident that only the sensitivities Vh(u) need to be known”. 


Beyond the conceptually simplistic idea of continuous-time gradient flow, there are different 
approaches to dealing with possible additional input or output constraints of the form 
uel’, (3.93a) 
yey, (3.93b) 
where U’ and Y’ denote the sets of feasible inputs and outputs, respectively. First of all, by 
defining U := U’ N h~'(J”), (3.91)-(3.93) can be written as 
min J(u) (3.94a) 
U 
subjectto wel. (3.94b) 
The approaches discussed in the literature differ in their strictness of constraint enforcement, 
in particular, whether or not each closed-loop equilibrium is always an “exact” optimizer of 


(3.94), and whether or not temporary violations of the constraint (3.94b) are allowed. Table 3.2 
gives an overview of the main research branches, which are briefly discussed in the following. 


43 Interestingly, the main impetus for the renaissance of this general control scheme came from the power system 


community, see e.g. [DBSPG19] for a historical perspective. The main reasons for their popularity in power 
system applications are their robustness properties as well as their ease of implementation with relatively low 
computational effort for the individual controllers (cf. [CSPB19]). 

Trivially, if J only depends on the input, the gradient descent simplifies to ù = —V J(u). For optimization 
problems which solely rely on y, see [JLv09, BDE12]. 


44 
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Table 3.2: Constraint enforcement schemes for continuous-time feedback optimization. 


transient constraint violation no transient constraint vi- 


olation 
no asymptotic + augmented gradient flow with + augmented gradient 
optimality penalty function [SDS88] flow with barrier func- 


tion [HBHD21a, p. 10f.] 


asymptotic op- + primal-dual saddle-point flow + projected gradient flow 


timality [Kos56, AHU58] [HBHD 16] 

e augmented saddle-point flow + projected Newton flow 
[Pol09a] [HBD21] 

* projected saddle-point flow 
[FP 10] 

e mixed saddle-point 
flow [HBHD21a, p. 14] 

« Newton saddle-point 


flow [HBHD21a, p. 12] 


Augmented Gradient Flow 


By augmenting J(u) with a scalar penalty or barrier summand U(w) indicating constraint vio- 
lation, the constrained optimization problem (3.94) turns into the unconstrained optimization 
problem 


min J(u) + O(u). (3.95) 


If the feasible domain U in (3.94b) can be characterized via inequality constraints of the form 
g(u) < 0, then for instance, U(w) = T||min{0, g(u)}||?, 7 > 0 provides a suitable penalty 
term and U(w) = -rIn(g(u)), T > 0 can be used as barrier term. As T — x, possible 
constraint violations become more pronounced in the augmented objective function, and 
thus constraint satisfaction is imposed more forcefully. However, while penalty and barrier 


functions always impose some sort of outer and inner approximation of the feasible set, 
respectively, the augmentation of J (u) typically leads to steady-state deviations between the 
closed-loop equilibrium and the optimizer u* of (3.94), especially if this optimizer is located 
at or near the boundary of U. But at the same time, 7 must still be sufficiently low to ensure 
a reasonably timescale separation between a “fast” plant system and “slow” optimization 


dynamics. 


Projected Gradient Flow 


A refinement of the gradient flow was introduced by projected gradient flow [HBHD16]. The 
remarkable feature of this method is that u(t) € U can be guaranteed at any time t, thus all 
constraints are always fulfilled even during transients. This is accomplished by requiring that 


64 3 Optimization-Based Control of Cellular Power Networks 


all directional changes of u should point into the feasible set. For this purpose, the right-hand 
side of (3.92) is replaced by the gradient projection onto U, leading to 


ù = —Ily(—VJ(u)) = — arg min || — VJ (uw) — £l], (3.96) 
geTu 

where Tu(u) := cl{d : d = a(w — u),u’ EU,a > 0} is the tangent cone of U at u. By 
definition, if u € intU, then Tu(u) = RP, while if u € bd, then Tu(u) represents the 
set of all vectors starting from u and pointing into the feasible set U. The ability of strict 
transient constraint enforcement is a salient feature of projected gradient flow. In [HBHD21b] 
quantitative statements about asymptotic stability of the closed-loop system with projected 
gradient are provided, which, however, require a priori knowledge of appropriate Lipschitz 
constants of the system with respect to both the input u and the state x. 


However, despite these advantages, projected gradient flow exhibits some major drawbacks, 
which hamper a practical application, especially in the light of the electricity market game 
mentioned in Subsection 3.3.5 with its intermittent nature of power consumption and/or 
production and a possibly large amount of distributed agents: Besides the fact that explicit 
expressions for the gradient projection (3.96) are often hard to find and generally not amenable 
to a distributed implementation, the main practical drawback of projected gradient flow is its 
tendency to “false equilibria” (i.e. equilibria of the closed-loop system that are no optimizers 
of (3.94)), whenever the current solution x(t) is infeasible with respect to (3.94b) at some 
point in time. We illustrate this effect by means of the following non-example: 


Example 3 (Projected Gradient Flow) 


Consider the optimization problem 


1 
min xe (3.97a) 
subjectto w+10<0. (3.97b) 


Note that (3.97) is a convex optimization problem with strictly convex objective function. 
Hence, u* = —10 is the unique optimizer of (3.97). WithU = {u E R : u < —10}, we get 


Reo, u> —10, 


3.98 
R, u < —10. oe) 


| 


Now consider the trajectory of (3.96) starting at uo = 10. With Tu (10) = R, we get 
ù = argmin || — u — €|| = -u. (3.99) 
EER 
Accordingly, a solution is given by u(t) = 10 exp(—t) and thus lim~o u(t) = 0, which 


means that u(t) never converges to its optimum value u* = —10. Trivially, the same result 
holds for any other u starting in the positive orthant. 


With respect to the CCs’ balance constraints (3.90) for instance, this shortcoming of projected 
gradient flow is highly problematic, since temporary constraint violations e.g. due to sudden 
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load jumps are always possible, potentially causing the resulting closed-loop equilbrium to no 
longer satisfy (3.90). 


Saddle-Point Flow 


Besides penalty methods and projected gradient flow, another means for incorporating con- 
straints is given by primal-dual saddle-point flow, originally proposed in [Kos56] and [AHU58]. 
It can be applied to optimization problems in standard form (cf. Definition A.18). The naming 
of primal-dual saddle-point flow stems from the fact that each optimizer of such optimization 
problems is a saddle point of the respective (optimization) Lagrangian. In particular, if (3.94b) 
can be represented by a set of r € IN U {0} equality and s € IN U {0} inequality constraints 
of the form h; (u) = 0 and ha(u) < O, respectively, then the Lagrangian of (3.94) takes the 
form L(u, A, u) = J(u) + A'hı(u) + w"ho(u), where A and ps denote the vector of dual 
variables for hı (u) and ha(u), respectively. Provided that some constraint qualification is 
fulfilled, the Karush-Kuhn-Tucker (KKT) conditions (cf. Definition A.20 in Appendix A.5) 
constitute first-order necessary conditions for the primal-dual optimizer (u*, A”, 4*) with 
gradient descent for the primal variables u and gradient ascent for the dual variables A, 


yielding 
OL (u, A, u) 


ù= -— i (3.100a) 
Ou 
A= 4, (3.100b) 


The inequality constraints ha(u) < 0 translate into complementary slackness condition 
(A.20c) and the nonnegativity of yz (cf. (A.20d)). Therefore, in [FP10], an additional projected 
gradient flow for the dual variables u(t) onto the tangent cone of the nonnegative orthant 


ju = + ns ,(V-2(u)) = +r (ho(u)) (3.101) 


is used to enforce u(t) > O for all t. Since the projection of hı,, € {1,...,s} onto the 
tangent cone Tr, allows for the explicit expression 


hy j(u ; hi > OV u; > 0, 
Tirso (hialu)) = (hih = (u): Mi, A (3.102) 
0, otherwise, 
the dual ascent (3.101) of u can be written compactly as 
ju = (hi(u)) i, (3.103) 


where (+), := Colie(1,...,5}{(-) ju} 


In contrast to the “pure” projected gradient flow (3.96) without dualization, this projected saddle- 
point flow (3.100), (3.103) yields a constraint satisfaction only at steady-state*’. Particularly, 


4 Despite this temporary constraint violation of primal-dual saddle-point flow, it should be emphasized that the 


resulting controller always yields “physically feasible” results w(t) at all times t, since the actual constitutive 
equations of the plant system are exactly respected at all times due to the feedback interconnection and thus do 
not need to be part of the constraints, as discussed above. 
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even the gradient projection (3.102) does not prevent temporary constraint violations of 


Despite this tendency towards temporary constraint deviation, however, especially in recent 
years, several control approaches in the area of networked and power systems rely on primal- 
dual saddle-point flow and its derivations (see the list of most prevalent methods in Table 
3.2). Likewise, some of the real-time economic dispatch and optimization-based frequency 
control approaches discussed in Subsections 2.2.2 and 2.2.4 resort to one of the above methods 
or a combination thereof for the real-time feedback optimization of their arising optimization 
problems. [MZL14, LZC16, TBD16, SDv17a, CY17] use the primal-dual saddle-point flow for 
optimization-based frequency control of S and/or £ nodes. [SDv17b] incorporates additional 
inequality constraints on active power injections and power line capacities (for networks 
with tree topology only) by introducing additional projected gradient ascent (3.103) for the 
dual variables. Moreover, additional penalty terms are proposed allowing an extension to 
non-strictly convex objective functions. [HZB* 17] uses projected gradient flow in conjunction 
with a penalty-augmented Lagrangian for a centralized voltage regulation with congestion 
management. The main reason for the popularity of primal-dual saddle-point flow approaches 
is their ease of implementation in contrast to the projected gradient method, especially if 
the underlying plant system exhibits a sparsity pattern. As discussed e.g. in [FP10, p. 1979], 
primal-dual saddle-point flow lends itself to a distributed implementation, which is highly 
desirable in view of the individual stakeholders defined in Section 3.3. Example 3 also stated 
and highlighted the “warm start” problem of projected gradient flow: Ifthe controller variables 
are initialized as infeasible or if individual variables become temporarily infeasible due to 
external events, the closed-loop system tends to invoke false equilibria. By contrast, primal- 
dual saddle-point flow can always be initialized even if the current state (w, y) does not 
satisfy all of the constraints. Moreover, the resulting controller equations (3.100) (and (3.103), 
if desired) always exist in explicit form, thus encouraging an automated controller design. 


Remark 3.27. There already exists a variety of hybrid forms between penalty, projected gradient 
flow or saddle-point flow methods (see Table 3.2). For instance, [Pol09a] suggests that further 
augmentation of Z (u, A, p) by another penalty term (resulting in the definition of augmented 
saddle-point flow) may help in damping oscillations, yet at the price of losing optimality with 
respect to the original problem. [HBHD21a, p. 14] proposes a mixed saddle-point flow where 
some of the inequality constraints are enforced directly by (gradient) projection and some are 
enforced by projected gradient ascent of the dual variables as in (3.103). © 


Remark 3.28. In addition, the right-hand sides of the controller equations (3.92), (3.96), or 
(3.100) can be premultiplied with any positive semidefinite matrix A(w) without changing the 
equilibrium. The prominent choice A(u) = (V2J(u))~4, yielding a Newton-type gradient 
descent with improved convergence properties has thereby given rise to the formulation of Newton 
gradient flows, projected Newton flow, and Newton saddle-point flow [HBD21, HBHD21a]. % 


With respect to practical application to electricity markets, a downside of augmentation 
methods and variations of saddle-point flow is that it usually sacrifices the distributed im- 
plementability mentioned above. In the light of this reasoning, particularly since temporary 
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constraint violations in the optimization problems (3.84) and (3.90) are tolerable according to 
the definition of power system stability presented at the beginning of Section 2.2, we will use 
projected saddle-point flow (with classical saddle-point flow for the equality constraints and 
gradient-projected dualization of the inequality constraints) as the basic building block for 
the local controllers, though we will discuss possible extensions where advisable. 


3.4.2 Resulting Control Laws 


Before deriving specific control laws for WPs and CCs by means of projected saddle-point 
flow, we give two general statements with reference to the optimization problem in standard 
form from Definition A.18. 


Lemma 3.29. Denote by (u*, A”, 4*) an equlibrium of (3.100), (3.103), and suppose that 


the optimization problem (A.19) has a strictly convex objective function fo(a) and fulfills 
Slater’s condition (A.21). Then u* is a solution of (A.19). 


Proof. Trivially, each (w*, A*, w*) fulfills the KKT conditions (A.20) for (A.19). Since fo(u 
is strictly convex, the KKT conditions are also sufficient for optimality if Slater’s condition 
holds (cf. [BV15, p. 244] and Lemma A.22). Thus, each u* is a solution of (A.19) . 


Lemma 3.30. Let fo(w) be strictly convex. Then the projected saddle-point flow (3.100), 


(3.103) asymptotically converges to the solution of (A.19). 


Proof. See [CMC16] for a proof using an extended version of LaSalle’s invariance principle 
for Carathéodory solutions of discontinuous dynamical systems or [Goe17] for an alternative 
proof using maximal monotone mappings. 


Remark 3.31. Ifthe strict convexity assumption of fo(u) from Lemma 3.30 does not hold, 
the application of saddle-point flow may cause the closed-loop system to exhibit limit-cycle 
behavior [HOBD20]. However, as shown in [HOBD20], the asymptotic convergence results of 
Lemma 3.30 can be regained by a suitable augmentation of Z (u, A, p). © 


Using Lemma 3.29 and Lemma 3.30, we now consider the individual optimization problems of 
WPs and CCs. 
For WP 7 € W, the Lagrangian of (3.84) equals 


Lr =-P, + ae a Zr Par) + Warn (Per = Pen) 
Er Ms, (Y, = Us. x) a HS4 AU exe a W,) 
Tv pr (Urn aA Ut.) + Kirn (UI Er) Uz.) (3.104) 
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where /,.) „ denote the vectors of Lagrange multipliers for the inequality constraints (3.84b)- 


‘) 7 a ies 
(3.84d) and W, := colicyy, „uv, {Wi}, Ua = colievw „uv, {Wi}. Since (3.84) is convex and 
Slater’s condition is fulfilled, the KKT conditions specifying a saddle point of Z, can be 


applied to derive a necessary and sufficient condition for an optimizer of (3.84)**: 


D = VCH (Ph n) — Ant We — Me + Met (3.105a) 
D = pS + MS 4 9 (3.105b) 
0 = -Už tM — (VE (Uz,2)) obs + (VW,(Uz..))o Kst,  (3.105c) 
D = (Mg_.x) © (Py n Par) (3.105d) 
D = (rn) (Par Par): (3.105e) 
0 = (us_ x) ° (Ur — Ut,x), (3.105f) 

= (M340) ° (Ut, - Urn), (3.105g) 
D = (už n) ° (Uz, - Ur), (3.105h) 

= (U4 r) © Ur — Ur), (3.105i) 
O < US- m» M34) HI- m PE Mee Mgt (3.105j) 


where VCr (Pg r) = Colicvy,, {VCg,i(Pg,i) }- Since no S’ node is adjacent to any Z node 
(cf. Subsection 3.2.5), it holds that VW, (Uz,,) = VW,(Uz,,) = 0, thus application of the 
projected saddle-point flow (3.100), (3.103) yields the controller equations 


TgnDg x = —VCn(De n) + An — Wa + Ugg — Met ye (3.106a) 

TU; Ute =Us_n — Hstn (3.106b) 
TUZI,» Uz. = UI- Mita (3.106c) 
Tann = (P, p T Per) ten’ (3.106d) 
T pgs x bgt = (Per — Per) pern? (3.106e) 
Tus- Msn = Us, —Utm) is p? (3.106f) 
Tysycbs+n = (Ute - Urdu, (3.106g) 
Tuz „Ër = Urs -Urn) i: (3.106h) 
Tuz, „Hirn = Ur - Urm)y,,, (3.106i) 


with suitable diagonal matrices 7.) > 0. The individual strictly positive diagonal elements in 
T(> are design parameters to be chosen by the individual WP. As outlined in Remark 3.28, 
premultiplication of the basic controller equations (3.100), (3.103) with an appropriate metric 
is helpful to adjust the convergence speed towards the equilibrium. The diagonal structure of 
T .) helps maintain a specific transient behavior for each primal or dual variable in (3.106) 
separately. In particular, the speed of the controller (along with possible overshoots) increases 
as ||7|| — 0. With regard to the practical application, each individual diagonal entry in T must 
thus be sufficiently high to ensure that the respective actuator is capable of performing the 


46 cf. Definition A.21 and Lemma A.22 in Appendix A.5. 
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required work’. At the same time, it is reasonable to assume that no WP has an incentive to 
set 7.) too high, since this would lead to an unnecessarily slow attainment of the own profit 
maximum. Besides these practical considerations, which are further discussed in Chapter 4, it 
is obvious that the specific choice of T.., does not alter the equilibrium of (3.106), cf. Remark 
3.28. 


The CCs’ objective P.(A%) = 0 from (3.90) can be accessed by means of the gradient flow 
Ak = —VP;(Ar), yielding 


Àr =- XO pest JO Pan (3.107) 
iEVZ,k iEVZ,k 
igVe 


The dynamics of the zonal price A; arise from the momentary active power balance of cell k 
through aggregation of all net power flows from cell k to neighboring cells. Illustratively, the 
price for power injection in cell k increases if the cell’s momentary production is lower than its 
own demand caused by RPs and transmission losses, and vice versa. Yet, instead of aggregating 
all required information at a central point, (3.107) can be set up in a fully distributed manner 
by defining a consensus protocol via a cell-specific, weakly connected communication graph 
Gr=(Vzx; Eor): In particular, let De, denote the incidence matrix of %. p. Then with 
Lemma B.1 from Appendix B.2, (3.107) can be implemented in a distributed fashion by 


Ta kÀk = —Pg,k + Prk + Pek — Deve: (3.108a) 
Tu nV = (Dex) Ar, (3.108b) 


where A, = col;ev; {Ai} and py = coljev, p {pi} with 


Pi = GuU? + > GijUiU;j cos(d;;). (3.109) 
JENG 
As above, T\,k and T,,x are real diagonal matrices of appropriate sizes with positive diagonal 
elements containing design parameters of the corresponding local controller. 7 ;, allows 
to adjust how fast-responding momentary power imbalances reflect in the prices, while the 
convergence speed of nodal prices A, towards a single cell-specific price A% can be adjusted 
via Ty, k- 


3.4.3 Analysis of the Closed-Loop System 


By virtue of using projected saddle-point flow, the initial state of the closed-loop system 
is not required to be located within the feasible set. However, in order to be able to attain 
an equilibrium, the following feasibility assumption must be made for the collection of 
optimization problems of WPs: 


47 Within the controller equations (3.106), the controller parameters Tg,” for active power injection are set to a 


comparatively high (i.e. “slow”) value, whereby further differentiation of T,,r in (3.106a) depends of the type of 
governing systems: While steam turbines of bulk CPPs require long ramp-up times and thus a very large T, .), 
the respective parameters for inverter-interfaced DERs may be set to rather moderate values. By contrast, the 
parameters Ty) y for the gradient ascent of dual variables can always be adjusted to a very fast value, as the 
dual variables correspond to “virtual” quantities that do not have to be generated physically. 


48 see Section B.2 in the appendix for some basic derivations. 
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Assumption 3.32. The upper and lower bounds in (3.83b)-(3.83c) are chosen sufficiently 


loose such that there always exists a feasible solution that satisfies the constraints for all 
WPs r € W. 


Let d* = col{q}, pý } be a given, constant disturbance input vector. We assume that the state 
vector £p of the physical plant system (3.49) fulfills the following regularity condition: 


Assumption 3.33 ( [TBD16, SDv17b]). The Hessian of H,(x,) is positive definite at 


* 
steady state x}. 


Assumption 3.33, which is commonly known in energy-related literature as existence of high- 
voltage solution [DMSP17], can be ensured by satisfying e.g. the (relatively mild) condition 
presented in [SDv17a, Proposition 9] and originally derived in [DM16, Proposition 1]. The 
condition is satisfied if all voltage angle differences ;; are small and if all generator reactances 
Xai Xi are small compared to the shunt susceptances b;. 


The simultaneous execution of the optimization schemes (3.106) for WPs m € W and (3.108) 
for CCs k € Z leads to a superposition of the derived control equations (3.106) and (3.108). 
Consequently, the resulting closed-loop system consists of the physical system (3.39) along 
with the controller equations 


TeP, = -VEP,)FA- We + Hg- — Hgt» (3.110a) 

Tu, Us = Hs- — Hs» (3.110b) 
TU ÜT = Ur — Mr, (3.1100) 
Ty, Mg = (P, — Pe), > (3.110d) 
Ty, Mgy = (Pg — Po) pyy? (3.110e) 
Tuss- = (Us —Us) ih, , (3.110f) 
Tus s+ = (Ur -Utis > (3.110g) 
Tu, Mz. = (Uz -Ur),,: (3.110h) 
Tuz Ära = Ur-Un),,, (3.1101) 
TÀ = -Py +P + pe- Du, (3.110)) 

Tub = DA, (3.110k) 


where the parameters and variables in (3.110) are stacked vectors or diagonal matrices of 
appropriate sizes for alla € W and k € Z, e.g. Tg = diagrew {Tgn}, De = cokkez{De,x}, 
and C(p,) equals the sum of all cost functions C+ (Py r) in (3.79). 


Remark 3.34. Note that D. has a block diagonal structure. Thus, the corresponding communi- 
cation graph GY, is disconnected and contains nz connected subgraphs. © 
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Analysis of the Closed-Loop Equilibrium: Optimality and Zero-Frequency Deviation 


At first, we point out a key result concerning the zero-frequency deviation of the closed-loop 
system (3.39), (3.110): 


Lemma 3.35 (Zero Frequency Deviation). At each equilibrium of (3.39), (3.110), it 


holds that w¥ = 0 for alli € V, ie. each node is operating at the nominal frequency. 


Proof. Eq. (3.39a) can be written in vector-matrix notation as v= Dw, where D, denotes 
the incidence matrix of the physical network %,. Since % is a weakly connected graph, 
each equilibrium w* with 0 = D)w fulfills w* = w* - 1, i.e. the nodal frequencies are 
synchronized at steady state. Since 1'p = ® and 1" D. = 0, left-multiplying (3.110j) by 1° 
and inserting the equilibrium values implies 


0=- X pt, t+ > pei +8. (3.111) 


iE Vg iev 


Due to the fact that ® = ),.,, Pi, a summation of all equations in (3.39b) and (3.39d) and 
insertion of the equilibrium values leads to 


= > Aw; + D Va X pei — ©. (3.112) 


i€ Vg i€Vq iev 


Comparison between (3.111) and (3.112) yields 


0=- Y Aw} = -u*- SO A. (3.113) 


iE Vg a 


Finally, since A; > 0 holds by definition, it follows that w* is zero. 


Remark 3.36. As the control equations of WPs are absent in the proof of Lemma 3.35, zero 
frequency deviation of the closed-loop system can still be maintained in the presence of clock 
drifts [Sch15, Remark 5.2.11] of the frequency measurement units of WPs (where the real nodal 
frequency w;(t) in (3.79) is pertubed by an additive disturbance +Aw,;). © 


The next lemma states that in each equilibrium of (3.110) marginal costs are equal at each 
node in a cell whenever the power injection constraints (3.80) are not binding. 


Lemma 3.37 (Equal Marginal Costs). Let x* denote an equilibrium of (3.110) and let 
k € Z. Then, at any node i,j € Va,r with p% ; € (P, i Pe,i) and p% ; € (P, p? Pe.) the 


marginal costs are equal, i.e. VCg i(pP3 i) = VCg,j (P4 3). 
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Proof. With (3.110k), it holds that © = DIA” at steady state. Since De = diag{D.,,} and 
since each De x is the incidence matrix of a weakly connected graph, this implies that each 
row of Aj has the same value, i.e. A% =: A; : 1. Moreover, from (3.110d) and (3.110e), we get 


+ 


Vu PN =O. (3.114) 


8 


If p3; € (P, p Pg,i) then both brackets in (3.114) contain negative arguments, which implies 
that Ho; = Hg+ i = 0 (cf (A.1)). Now, with w* = 0 from Lemma 3.35, it follows via (3.110a) 
that VC,.,(p2;) = A} = Ar. Hence, all marginal costs are equal to the cell-specific value 
Ar. 


A trivial implication of Lemma 3.37 is that the special case of quadratic cost functions 


@;>0, i€ Ver, (3.115) 
2 Wi 


Cy i (Dg,i) = 


yields active power sharing in cell k € Z, which is defined as follows: 


Definition 3.38 (Active Power Sharing [SOA* 14, p. 2465]). Let wp = coliev, p {0k} 
> 0 denote a vector of constant weighting factors. Then active power sharing applies in 


cellk € Z if 


Pei Pag u 
Er = 8) i,j € Vek (3.116) 
Tri Wk; 


Shifted Passivity and Stability 


Since the physical plant system is amenable to a port-Hamiltonian representation (3.49), it is 
convenient to evaluate the closed-loop stability based on passivity arguments. As discussed 
in [van17, Section 4.4], PHSs can be decomposed into an interconnection of smaller subsystems 
to simplify passivity analysis. For the sake of clarity, we thus conceptually segregate (3.110) 
into a frequency controller (3.110a), (3.110d)-(3.110e), (3.110j)-(3.110k) and a voltage controller 
(3.110b)-(3.110c), (3.110f)-(3.110i) and analyze each of these components separately. 


Lemma 3.39. The physical plant system (3.49) with output Yp = Gl zp is (locally) shifted 
passive with respect to £$, if 


[zp - a [R(ap) - R(zt)] > (3.117) 


with R(&,) = Rpzp + rp holds. 
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Proof. The physical plant system is (locally) shifted passive with respect to x}, if the shifted 
plant Hamiltonian [van17, p. 136] H,(&,) := Hp(zp) — (&,)" VA,(25) — Hp(x}) with 
Lp = £p — LF satisfies H,(&,) > Oand 
Hy (Zp) < Yr p (3.118) 
; _at 
with yp = G, Zp. 


The positive definiteness condition is satisfied locally due to Assumption 3.33. To investigate 
(3.118), let the disturbance input d* be constant in time and define R(z,) = RpVHp(£p)+rp. 
Then (3.49) can be written as 


Et, = JpVHp(ap) — R(ap) + Gpup + Gad* (3.119) 
with each equilibrium xj fulfilling 
0 = J,VH,(25) — R(z}) + Gpus + Gad* (3.120) 


for a constant input vector us. Since the shifted Hamiltonian H, p(p) is positive definite with 
minimum H,(0) = 0, the shifted plant dynamics, i.e. (3.119) minus (3.120), can be expressed 


in terms of H,(&p) as follows: 
Ex, = JpVHp(xp) — [R(£p) — R(z})] + Gp%,. (3.121) 


With H p(&p) = ZI EX, and bearing in mind that J, is skew-symmetric, the passivity 
condition (3.118) is equivalent to (3.117). 


Corollary 3.40. Ifthe physical plant system (3.49) with output y, = GI zp is lossless, 


then it is (locally) shifted passive with respect to x}. 


Proof. For lossless grids, it holds that r, = 0. Hence, (3.117) is always fulfilled due to the fact 
that R, > 0. Accordingly, (local) shifted passivity of (3.49) follows from Lemma 3.39. 


Next, we examine the shifted passivity of the frequency controller (3.110a), (3.110d)-(3.110e), 
(3.110j)-(3.110k): 


Lemma 3.41. The frequency controller (3.110a), (3.110d)-(3.110e), (3.110j)-(3.110k) with 
input u. = —w, and output y,,; = Pg is (locally) shifted passive, if 


(Pg — px)" (VC(p,) — VC(p})) 2 (A— A*) "(9 — p*). (3.122) 
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Proof. By defining the frequency controller state &| = col{T gpg, Tu, Mg, Tp, Hgy TAA; 
Tv} and the frequency controller Hamiltonian Hj (€,) = lel role, with Te = diag{ Tg, 
Tug Tug Tas Tv} > 0, (3.110a), (3.110d)-(3.110e), (3.110j)-(3.110k) can be written in port- 
Hamiltonian form 


& = JıVH+(E,) =f} + Guru: + Gapy, (3.123) 
Yu = Guilı, (3.124) 
Ya = Gng, (3.125) 
where C = VH (£1) = col{P,, Ha, Hg+, À, V} and 
0 00 Tf, 0 Vc(p,) ] 
0 00 (0) 0 (P, = Pg) 
ae ag De ie ri = |- (Ps — Po) | > (3.126) 
-I, 00 0 D, —p 
0 00 -D}? 0 0 
I 0 
0 0 
Gi = lol, Ga = |0 (3.127) 
0 I 
0 0 


With the shifted controller Hamiltonian 
Hy (€) = Hy (€,) - (E1) "VH (Et) - Ha (€i), (3.128) 
the shifted controller co-state G equals 


Či = VÄ (E) = VA (é) — VA (64) = 61 - CF. (3.129) 


Since the disturbance input p, is assumed to be constant, (3.123) can be expressed in shifted 
coordinates as 


& = AHV) — [r1 (61) — (ED) + Ga, (3.130) 
Ja = Gagi- (3.131) 


Due to strict convexity of H; (£4 ), positive definiteness H, (€) > 0 is always satisfied. Hence 
the frequency controller is (locally) shifted passive if 

Lo ~~ at ,T i -T a LATS 

Ale) = VE) Ale) FE] + a Gurte < Hire. (6.132) 
By inserting (3.124), we get 

eo! * * * 
0 > G,[rı(,) - ri (éi) = —(p, - pi)" (VC(p,) - Ve(p})) 

23 Dy Mg aa Dy Py + js (P, E Pe) ps + TER (Po = Peek 
+ (A- A) pe- p*). (3.133) 
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In [SDv15, Proposition 3], it is shown that u (Vi < ag and u g* < 0 hold for each convex 
: =T T ~T u 
function g = col;{gi}. Hence A, (P, — Pe), S Mg- (P, — Pg) = Fg- (P, — Pz — Pg) < 
EA With the same procedure it can be calculated that ru (Pe — Da) ji es +s. Pg 

8 
Thus, the second line of (3.133) is less than or equal to zero, which leads to (3.122). 


Corollary 3.42. Ifthe plant system (3.49) is lossless, then the frequency controller with 


input u. = —wg and output y,,; = Pg is (locally) shifted passive. 


Proof. For lossless networks, it holds that p = 0, hence (3.122) in Lemma 3.41 is always 
fulfilled due to the fact that C(p,) is strictly convex by Assumption 3.12. 


Remark 3.43. Since the interconnection between frequency controller and physical plant system 


zi (3.134) 


5 


up = Ya = GV Ai (£) = P, (3.135) 


uci = -Yp = -G VH, (£p) =W 


is power-preserving, i.e. uliya + U) Ypı = 0, shifted passivity of the subsystems in terms 
of Lemmas 3.39 and 3.41 implies shifted passivity of the closed-loop system (3.49), (3.110a), 
(3.110d)-(3.110e), (3.110j)-(3.110k). In fact, the conditions stated in Lemmas 3.39 and 3.41 are 
only sufficient and not necessary, as an excess of passivity in one subsystem can compensate for 
the lack of passivity in an other subsystem, cf. [van17, p. 23]. © 


Based on the previous conditions we now formulate a stability criterion for the lossy, physical 
plant system along with frequency and voltage controller: 


Theorem 3.44. Assume that the conditions of Lemmas 3.39 and 3.41 hold. For a constant 
input d*, let (£$, £7, E3) denote an equilibrium of (3.49), (3.110). Then there exists a neigh- 


borhood B around (x¥, E1, E3) such that if (£p, 1, 2) € B, then the state asymptotically 
converges to (x$, 1, 5). 


Proof. With the voltage controller state & = coH T,,_HUs-; Tus Kst TU; Ur, Ty, Mt; 


Tuz, Yr+,Tu,ÜUr}, let €5 denote an equilibrium of (3.110b)-(3.110c), (3.110f)-(3.110i) and 
~ ~ ~T xT 

define the Lyapunov function candidate Vz(£3) = $£)77 €, where T2 = diag{T u, , 

Tus TU Tur Tas Tu} > 0 and &, = €, — €3. This allows to write out an overall 


Lyapunov function candidate V (Hp, Ei, Eo) for the overall closed-loop system as the sum of 
Hp, Hy, and Va, i.e. 


V (p E1 2) = Hp (čp) + A E) + VE) > 0. (3.136) 
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With respect to y, we observe that 


V (Bp, č ča) = Hp (Bp) + AE) + Vo), (3.137) 


where the first summand of (3.137) equals 


Hp (®p) = (V F(E) Ip (V Hp (p)) — (VÄ E)" [R(ap) - Rei] 


= —(Zp - z5)" [R(z,) — Rie})] (3.138) 


due to the skew-symmetry of J p. The second summand of (3.137) is 


if, (€,) = -PE (Vo(p,) — Velp) + (A-d*)"(p — PN). (3.139) 


The third summand of (3.137) equals 


00 5 _ = ~T ~T 
Valz) = ñs- (8 - Unk, + As (Ur - Bh, +U; ùs- -Ut is 
+ fiz_(Uz—Uz)h +, (Ur-Un),,; (3.140) 
In [SDv15, Proposition 3], it was shown that u (9) < hg and je g* < 0 hold for each 
convex function g = col;{g;}. Hence fis_ (YW — Uni, S ug (Œ — Us) = ùk (Y -— 
Ut- Ū;) < —fis_U;. With the same procedure it can be calculated for S+, Z—, and T+ 
that the first two rows of (3.140) are less than or equal to zero and thus Vs (€) < 0. With 
the assumption from Lemma 3.39, the first row of (3.138) is < 0 and with the assumption 
from Lemma 3.41, it also holds that H,(E,) < 0 in (3.139). Hence V (čp E1 E2) < Oin 
(3.137). Since V + OandV < 0, the discontinuous closed-loop system (3.49), (3.110) fulfills 
the condition of [CMC16, Lemma 4.3]. According to [CMC16, Theorem 4.5], the limit set 
of each solution of (3.49), (3.110) starting in B is a singleton, hence (xy, £7, €5) is (locally) 
asymptotically stable [Kha02, Corollary 4.1]. 


3.4.4 Summary and Illustrative Example 


In this section, we have introduced an optimization-based control scheme for WPs and CCs via 
projected saddle-point flow ensuring frequency and voltage regulation. While the latter can 
be ensured directly via local constraints (3.84b) and (3.84c), frequency regulation is achieved 
indirectly via the real-time price A by incorporating w in the profit functions of each WP”. 
The derived price-based control scheme provides a rigorous compliance with the unbundling 
paradigm: WPs as competitive players require only the nodal prices A; transmitted by the local 
CC individually for each node. Apart from this communicated quantity, the decision-making 
of WPs is fully decentralized. Namely, there is no need for the WPs to reveal their own cost 


# In contrast to this native implementation of the frequency control task, the optimization-based frequency control 


schemes presented e.g. in [TBD16, SDv17a, ZMLB18] artificially impose a frequency-dependent term as extra 
control input in addition to a saddle-point-based controller. 
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function Cy to competitors or CCs, where dishonesty by the WPs would be likely anyway 
(see e.g. [TJ13, p. 562]). In addition, a distributed realization can be adopted for the control 
actions of CCs as an outcome of the reformulation (3.108). The resulting local controllers then 
only need to have knowledge about the local values pg ; — pe; and U;, voltages Ü; of direct 
neighbors j € N;, as well as line parameters and communication variables 1;; of adjacent 
lines. Moreover, since no WP 7 € W can unilaterally increase its profit” P, whenever at 
steady-state p, , = Pr m each pz represents a Nash equilibrium of the multi-player game 
(3.83), cf. [BLMH* 21]. 


Due to the port-Hamiltonian model of the physical plant system (3.49) and frequency controller 
(3.110a), (3.110d)-(3.110e), (3.110j)-(3.110k) as well as the nonnegativity property [SDv15, 
Proposition 3] which is exploited in the proofs of Lemma 3.41 and Theorem 3.44, closed-loop 
stability can easily be assessed by passivity arguments. Even ifthe control scheme is certainly 
intended for lossy networks, a strict mathematical proof that provides an a priori statement 
on closed-loop stability is only available for lossless networks. This is due to the fact that 
the conditions (3.117) and (3.122) can only be evaluated online, yet are always fulfilled in the 
lossless case. This result is in line with the concern that there exists no analytical energy-like 
function for “generic” power grids with nonzero transmission losses, which has been addressed 
by many authors over the last eight decades [Mag47, Nar84, KBP85, ASB01, SALB05, VT15]. 
In fact, energy functions which enable a Lyapunov-based stability analysis are only available 
for very specific academic test systems [BA03, GSAB05, OGA* 05]. However, even for highly 
low-dimensional systems, there are counterexamples where all Lyapunov-like functions are 
not applicable [Nar84]. Yet an advantageous property of conditions (3.117) and (3.122) is their 
modular structure: If passivity could be verified for two smaller subsystems (3.49), their power- 
preserving interconnection”! always leads to a passive overall system, which potentially gives 
rise to a plug-and-play based approach to stability analysis, see [APM16, SP19, SMKH20] for 
more in-depth discussions. We illustrate some of the theoretical insights of Section 3.4 by the 
following academic example: 


Example 4 


In this example, we investigate the frequency stability and active power sharing of the closed- 
loop system resulting from the exemplary physical plant system from Fig. 3.8 along with the 
controller (3.110). For the sake of brevity, we choose G = (R/X)-B with (R/X) € NU{0} 
yet to be determined and set all power and voltage constraints equal to infinity. The units of 
all parameters are given in p.u. with Syase = 1 MVA and Upase = 10 kV. The parameter 
values of the plant system can be found in Appendix B.4. The controller parameters in (3.110) 
are all set to T, = 0.1: I and the cost functions are chosen to 


11 
Cg,i(Pg,i) = 5— "Pi (3.141) 


°° Figuratively, this is the case since at equilibrium, the profit margins of all players are zero. Indeed, if they 


deviate from their strategy (P$ ~, Uf „, UZ.,,), the new strategy would either be infeasible with respect to the 
constraints (3.84b)-(3.84d) or they would be worse off as the additional cost VC+ (Pg x) for power production 
exceeds the additional revenue VRr (Pg x )- 


>b >b >b >b 
51 e.g. by coupling all boundary voltage phasors U] , U > and power injections Pei P? 2 Qa: Qo viaU, = U3, 


P} a = =P} > and d?a = =a? os respectively. 
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with weighting factors 7, = 1,w2 = 1.1, w3 = 1.2, w4 = 1.3, and ws = 1.4. The network 
is initially in steady state with nominal frequency 9n, = 50 Hz and constant loads Pe i, qei. 
Att = 10s and t = 40s, a step load change of Ape i = +0.1p.u. occurs at load nodes 6 and 
7, respectively. 


Figure 3.8: Stylized representation of the exemplary physical plant system with ng = 2, nz = 3, and ng = 2. 
S nodes are depicted in black, Z nodes are depicted in gray, and £ nodes are depicted in white. Each 
bus is equipped with an uncontrollable load modeled by (uncontrollable) active and reactive power 
consumption. 


Fig. 3.9 shows that for R/X € {0,1,2}, all nodal frequencies converge to the nominal 
frequency within 10s after the load jumps. For R/X = 3, however, there is no synchronization 
of frequencies. 
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Figure 3.10: Active power injections for Example 4. 


For R/X € {0,1,2}, the active power generations pz; are equidistant for each post-fault 
equilibrium, i.e. active power sharing is maintained (see Fig. 3.10). As the R/X ratio increases, 
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also the final values of p, increase slightly. This can be traced back to the fact that the WPs 
have to compensate the increasingly high transmission losses. For R/ X = 3, as already seen 
for the frequencies, there is no convergence to a final value p3. 
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Figure 3.11: Dissipations in the system for ascending R/X ratios. The system becomes unstable for R/X = 3. 


As apparent in Fig. 3.11, a negative (shifted) dissipation (3.138) can be observed for R/X = 3, 
while (3.138) is nonnegative for R/X € {0,1,2}. The curve of (3.138) along with Figs. 3.9 
and 3.10, suggests that the momentary dissipation rate is a useful tool to qualitatively assess 
the stability margin of the closed-loop system. 

Now we reset R/X equal to 1 and assume that the measurement of w at node 1 constantly 
deviates from the actual frequency by +0.1 Hz to simulate a clock drift. 
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Figure 3.12: Nodal Frequencies and active power injections in the presence of clock drifts. The red lines indicate 
the nodal frequency and active power injection at node 1. 


As depicted in Fig. 3.12, simulations show that the controller is robust in terms of clock drifts 
and is able to restore the nominal frequency Qn i (cf. Remark 3.36). However, active power 
sharing can no longer be achieved by node 1, which is in line with the findings from [Sch15, 
Remark 5.2.11] that clock drifts affect the ratios (3.116) of active power sharing. This can 
easily be verified by the fact that Lemma 3.37 concerning equal marginal costs is violated in 
the presence of clock drifts. However, as argued in [Sch15], the introduced quantitative errors 
in power sharing are negligible for most practical applications. 


Remark 3.45. The fact that variations of Us ; or Uz; typically come at no additional cost for 
the respective WP (cf. Remark 3.22) can be exploited by imposing additional responsibilities on 
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the voltage controller (3.110b)-(3.110c). For instance, the augmentation 


ry,Us = -Us + US" + us — Ms, (3.142a) 
Tu Uz = -Uz + U8 + pr — ey (3.142b) 


can be employed to follow a specific setpoint U;* = coliev, {Ui (UF")}, Us = coliey, {US}. 
Alternatively, to also achieve reactive power sharing [SFO15, p. 60] 
* * 
fsi Tsj i,j E€ Vg (3.143) 
Xi Xj 
with qg i = qinj,is g,j = qinj,j and Xi, Xj > 0, the distributed consensus-based control scheme 
from [Sch15, Section 5.3] can be applied” if the network 9, is mainly inductive (G * 0), yielding 


Tupi Üti > —k; (= = tes T MS— ji T MS+,i> tE Vs; k; > 0, (3.144a) 
JEN, 


Tuz:ÜZ,i = —k; (= _ tes ) + Ur i — HIti; i € Vz, ki > 0. (3.144b) 
JEN, 


Note that even with one of these augmentations, steady-state compliance with the voltage ampli- 
tude constraints (3.84c)-(3.84d) is still preserved due to the projected gradient flow of the dual 
variables (3.110f)-(3.110i). o 


3.5 Interconnection of Zonal Prices 


Lemma 3.37 has revealed that at each equilibrium of (3.110), nodal prices X;, A; € Vz,k 
are equal to a common zonal price Ar for each cell k € Z. Incentives for increased or 
decreased power generation are thus imposed implicitly by means of differences in zonal prices. 
However, Remark 3.34 implies that there is no direct relationship between each individual 
Ar to be adjusted. Therefore, in the following Subsection 3.5.1, specific couplings between 
CCs are introduced that enable such relationships. Subsequently, Subsection 3.5.2 reveals the 
connection of the resulting closed-loop equilibrium with a (notional) centralized optimization 
problem. On this basis, Subsection 3.5.3, which contains the main result of Section 3.5, 
evaluates the economic efficiency of this closed-loop equilibrium as a first milestone towards 
Contribution 2. 


3.5.1 Coupling of Zonal Prices 


To provide interdependencies between the zonal prices, additional constraints of the form 


Ài = Nij © Àj, (3.145) 


52 For the sake of simplicity, we neglect the filter elements for measured active and reactive power injections Pg,is 


qg,i Originally included in the controller equations of [Sch15, Section 5.3]. 
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with an appropriate multiplier 7;; > 0 can be imposed for pairs of nodes 7, 7 € V which are 
located in different cells. For this purpose, the equilibrium condition of (3.110k) along with 
the linear dependencies (3.145) can be written compactly as 


0 =(Dt)'d. (3.146) 

Hence, the resulting controller equations (3.110j)-(3.110k) are given by 
Ta = =P; + P + Pe — Driv, (3.147a) 
Tb =(Dt)'X. (3.147b) 


With this notation, Dt can be interpreted as the incidence matrix of an extended, weighted 
communication graph + = (V, (Ee, EP)), where EP represents communication across cell 
boundaries. The weights of the edges of Z+ are equal to 7;;, if the two adjacent nodes belong 
to different cells, and equal to 1, if ¿ and j are located in the same cell. 


Assumption 3.46. The multipliers nij are chosen in a feasible sense such that dim ker( D} )" 


> 0. Practically speaking, we assume that there exists at least one X > O fulfilling (3.146). 


Remark 3.47. Assumption 3.46 is fullfilled if and only iflog{n} € im(D.). In this case, each 
cellk € Z can be characterized with a specific participation factor x, > 0 such that each ni; is 
calculated by nij = Kr, /Kr, ifi is located in cell kı and j is located in cell ka. © 


Lemma 3.48 (Connectivity of Zonal Prices). If is weakly connected, then 


Aı Ag 


RKkı Kko 


(3.148) 


where Kx, denotes the corresponding participation factor belonging to the cell k where node 
i € V is located. 


Proof. Define the auxiliary matrices Kı = diag; jce,,ev) {mij} > Oand Ko = diagyey{kx, } > 
0. Then, (DF)TA = 0 is equivalent to 


K,'0 = KI (DV A = KI '(D})' K K3'A. (3.149) 
= N 
0 =:(D2)T =:2° 


Inserting the definition of Kı into (3.149) with nij = Kx, /Kr, reveals that D? is the incidence 
matrix of a new communication graph Y°, which is equivalent to Y.* with all edge weights 
reset to 1. Thus each solution A° of the resulting equation © = (D°)'A° is of the form 
A° = 1- const, i.e. each component of A° has the same value. Finally, since A° = Kyz'X = 
coljev{A;/kr,}, this leads to (3.148). 
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Remark 3.49. If$ is not weakly connected, then (3.148) holds separately for all nodes in 
each weakly connected subgraph inG*. © 


Remark 3.50. The uniform price A° resulting when x, = 1 for allk € Z refers to the social 
optimum (cf. footnote 35), yielding a minimization of the overall costs of WPs for power generation 
as well as equal revenues for the same amount of active power generation irrespective of where 
it is located. Accordingly, x, describes the multiplicity of the zonal price A, compared to the 
uniform price A°. © 


In the following, we will discuss to what extent « = colke z {Kp} can be applied to modify 
the impact of specific cells on the overall network. 


3.5.2 Comparison with Centralized Optimization 


Conceptions of an overall utility within competitive scenarios are strongly related to the idea 
of global efficiency, which was alluded to in Subsection 3.3.1. The information about which 
solution is considered “socially preferable” to which other solution can thus be expressed by 
means of welfare functions (cf. Appendix A.6). In this context, the next theorem reveals an 
interesting connection between the multiple optimization problems of WPs and CCs anda 
notional centralized optimization problem with a modified utilitarian welfare function which 
is a weighted sum of the individual WPs’ profits: 


Theorem 3.51 (Equivalence to Centralized Optimization). Define the centralized 
optimization problem 


max p* (3.150a) 
pz,U:,Ur 


subjectto ® = = Da = N pe,is (3.150b) 
iE Vg iev 


P, S Pg < Po» (3.150c) 
U,<U;<U,, (3.150d) 
Uz < Uz < Uz, (3.150e) 


Per = > = - Cg,i(Pg,i) — 5 Wi ` Pg,i- (3.151) 


keZ TEVERE CE 


Then each optimizer (p%, U},U7) of (3.150) is an equilibrium of (3.110a)-(3.110i), (3.147a)- 
(3.147b) and vice versa. If all Cg ;(Pg,i) in (3.151) are strictly convex, then (pj, 7,U7) is 
unique. 
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Proof. Constraint (3.150b) is equivalent to (cf. [SDv17b, p. 2615]) 
D.b = p, — Pe — P, (3.152) 


where p = col;{p;} as in (3.55)-(3.56) and D. is the incidence matrix of a weakly connected 

communication graph. If we choose D. = D? and define C” (Pg) = rez Lievz x . 
k 

Ci (Pg i), then the Lagrangian of (3.150) becomes 


PER REEN 
Z" = C" (p) + XO wi Pegi +A (Dev — Pg + Pe +p) 


iE Vs 
+ fig- (P, — Pg) + Ags (Pe — By) + As- (Us — Us) + Ês (Ur - Dr) 
+ Êz- (Ur - Ur) + Az; (Ur — Uz), (3.153) 


where denotes the Lagrange multiplier for equality constraint (3.152) and fu.) are the 
Lagrange multipliers for the inequality constraints (3.150c)-(3.150e). Since (3.150) is convex 
and Slater’s condition is fulfilled, the primal-dual optimizer of (3.150) is given by the KKT 
point (-)* defined by 


0 =—Ve" (pt) — wi + ia: fÈ- pl, (3.154a) 
0 = Ah- Ah (3.154b) 
0 = —Â3- + Mz, (3.154c) 
0 = (jik_)o (p, — pi), (3.154d) 
0 = (AË) o (pi — Pe), (3.154e) 
0 = (Ê$ _) o (U; — UN), (3.154f) 
0 = (A$) 0 (U$ — U;), (3.154g) 
0 = (à$_) o (Uz — U4), (3.154h) 
0 = (fh) 0 (UL -Up), (3.1541) 
0 = -p +p +p- Doi", (3.154j) 
0 =(D°)T A, (3.154k) 
0 < AS AS, AEA, ae AR. (3.1541) 


Inserting the definition (3.149) in (3.154k) and comparison with the right-hand side of (3.110k) 
yields = K3'A*. Hence with Vc" (PÈ) = Kz'Vc(p}) in (3.154a) and by comparison 


* 


between (3.154a)-(3.1541) and (3.110a)-(3.110k) we get the equivalences p; = p%, u! =U;, 
U} = U}, lig = Ky ye, gy = Ky hy, is = M3 Asy = MS, A = HE 
ji + = H44. Moreover, the dual optimizer v* from (3.110j) (using the notation (3.147a)) and 


the primal optimizer D” of (3.152) are connected via the relationship —D°v' + Div* =0. 
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From (3.149) it follows that D? = KoD} K7{' with Kı, Kə > 0. Accordingly, for each 
equilibrium A* of (3.110k) there exists a x fulfilling (3.154k), and vice versa. 


In summary, for each primal-dual optimizer of (3.150) there exists exactly one corresponding 
equilibrium of (3.110a)-(3.110i), (3.147a)-(3.147b). In particular, (pi., Ut, UF) = (ph, U!,U7). 
Since (3.150) is a convex optimization problem, convergence of the trajectory (p,(t), U:(t), 
Uz(t)) to (pz, Us, UZ) is guaranteed (cf. [Ant94, Theorem 2)). 


If the cost functions Cg ;(pg,i) are strictly convex, then P“ is strictly concave. Hence, the equilib- 
rium (pi, uf, ut) from centralized optimization and therewith the equilibrium (pz, Uf, Uz) 
from distributed optimization are unique. 


Remark 3.52. If« = 1, then the objective function of (3.150) is 


P= X Pr- So k+ YO Pr = Y Cilp), (3.155) 


TEP kez kez iEV 


and thus equal to the sum of all payoffs of WPs (3.79), RPs (3.88), and CCs (3.90). Accordingly, the 
distributed controller (3.110a)-(3.110i), (3.147a)-(3.147b) leads to a constrained minimization of 
the overall costs of power production. © 


3.5.3 Analysis of Pareto Efficiency 


As stated in Theorem 3.51, the interaction of distributed WPs and CCs, subject to cell-specific 
pricing via xx, leads to the same equilibrium (p}, Uf, U7) as if a centralized authority with 
full knowledge of the whole plant system would solve the optimization problem (3.150) with 
modified cost C“(p,), where the cumulative cost functions of each cell k € Z are divided 
by Kx. This equivalence reveals some further key properties of the developed distributed 
controller scheme that allows the evaluation of the Pareto efficiency of the equilibrium from a 
multi-objective perspective. 


Remark 3.53. With the same reasoning as in the proof of Lemma 3.35, it can be proven that 
for each equilibrium solution of (3.39), (3.154), it holds that w = O, i.e. zero frequency deviation 
is preserved. oO 


In particular, it can be shown that each x > 0 achieves an efficient allocation in the sense 
that for a given K’ > 0, there is no possibility to find a “better” k” Æ K’, K” > 0 such that at 
least one cell is at alower cost and no cell is at a higher cost. This is formalized in the next 
theorem. 
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Theorem 3.54 (Pareto Efficiency of xK). For each k € Z, consider the optimization 
problems 


max  ptotal (3.156a) 
Pg,kUf,k:UT,k 


subject to (DI) Ak =0, (3.156b) 
Da < Pek < Pek: (3.156c) 
Uy p < Use < Ue, 3.156d) 


( 
Uz, < Ure. < Utk, (3.156e) 


piotal _ DW = Pra) E i (3.157) 


iEVZ k TEP 


denotes the net profits of all network participants in cell k € Z and where D}, in (3.156b) 
contains all elements of (3.146) that belong to the nodes in cell k. Let kı > 0 be fixed 
and let the corresponding value of P‘°'*! for an equilibrium of (3.156) with parameter Kk 
set to Kı be denoted by P°t*'* (w1). Then, there exists no other dominating K2 > O with 
K2 # Ka such that the following two conditions hold: 


Vke2: Peg, Spa), (3.158a) 
the 2: Picea) De an) (3.158b) 


Proof. Taking into account that (3.156) are concave optimization problems and following the 
same lines as in the proof of Theorem 3.51, it can be shown that each KKT point of (3.156) is 
given by (3.154a)-(3.154]) and vice versa. For each x > O, the equivalent problem (3.150) is a 
linear scalarization of (3.156) with positive weights 1/«; > 0. With [ED18, Proposition 9], 
it is implied that for each k’ > 0, the solution of (3.156) is a part of the Pareto front, hence 
there exists no K’ > 0 which is dominated by another K” > 0 in terms of (3.158). 


The observation of Theorem 3.54 that any market equilibrium resulting from a given k > 
O yields a Pareto efficient allocation for players 7 € W is fully consistent with the first 
fundamental theorem of welfare theory (see Theorem A.25 in Appendix A.6). However, as 
opposed to the “copper-plate based” statements of Appendix A.6, Theorem 3.54 provides a 
statement for real-world electricity markets which are faced with operational constraints 
and a lossy power network. We illustrate the effects of different values of k by means of 
Example 4. 
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Example 4 (cont’d) 


Recall the illustrative 7-node example from Subsection 3.4.4. We now assume that the network 
is divided into two cells as depicted in Fig. 3.13a, with kı set to 1. 
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(a) Schematic diagram of cell partitioning (b) Scatter plot of (P{°tal, ptotal) 


Figure 3.13: Exemplary 7-node system divided into two cells. 


Fig. 3.13b shows a scatter plot of the resulting steady-state (P}°t*!, ptotal) after each of both 
load steps, and under different choices of k2 € [0.2, 20]. It can be seen that in both cases, each 
point (p{otal, ptotal) is part of the Pareto front, thus no ka yields a solution (P*°'*!, ptotal) 
which is dominated by another xy. 


3.6 Discussion 


In this chapter, we have elaborated a distributed intra-cell mechanism for optimization-based 
control in zonal electricity markets. The proposed mechanism allows a mixture of price-taking 
WPs, price-setting CCs, and inelastic RPs. The interplay of different optimization problems 
of WPs results in the overall control system (3.49), (3.110) for which frequency and voltage 
regulation is preserved. The entailed constraints for WPs are either technically motivated 
or used to incorporate other obligations of WPs, e.g. from day-ahead clearing or long-term 
contracts. In this regard, the feasible set of each WP’s optimization problem (3.83) contains 
the capacity that is currently available to the real-time market and not contracted otherwise. 
In line with the brownfield paradigm addressed in Chapter 1, WPs are not obliged to disclose 
their own profit function or their momentary flexibility to local CCs or to competitors. The 
proposed control scheme thereby provides a valuable alternative and improvement to other 
concepts currently under discussion, which are always characterized by an increasingly strong 
normative interventionism (see Subsection 2.1.3). 


Apart from the cell-based communication structure, all WPs compete in a common market, 
which helps reconcile individual profit and social welfare, thus limiting the moral hazard 
phenomenon (cf. [ZPE17, p. 180]). By pushing the market price to marginal cost level, the 
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allocation (P „, Uf.,U7,,) resulting as a closed-loop equilibrium is individually rational 
from the perspective of each WP m € W, since any other feasible allocation would lead to 
lower individual profits. Moreover, Theorem 3.54 guarantees that by permitting cell-specific 
couplings via some & > O, any equilibrium is also Pareto efficient. 


Despite these promising findings, however, Pareto efficiency is unsatisfactory if used as the 
sole solution concept, since no concrete statement has yet been attempted on the proper 
choice of particularly suitable or unsuitable x € RZ, which could theoretically lead to 
allocations with arbitrarily poor social welfare. In the next chapter, we shed light on such 
unique allocations by proposing different control schemes for the dynamic and distributed 
settlement of « throughout the interconnected power system. 


4 Real-Time Incentives by Zonal Pricing 


In this chapter, distributed control schemes for inter-cell pricing are developed, working 
towards Contribution 2 and 4 (see Fig. 2.3). Section 4.1 defines the requirements for the control 
framework to be developed. Section 4.2 gives an overview of the developed overall control 
structure along with the dependencies of the individual sub-controllers. Section 4.3 provides 
real-time control strategies for the automatic regulation of zonal price differences by means of 
kK in order to fulfill Contribution 4. Section 4.4 provides the main results for Contribution 2 by 
proposing a dynamic balancing controller which guarantees fair pricing from the perspective 
of RPs. Section 4.5 analyzes the overall closed-loop system resulting from the interconnection 
of the proposed controllers. Section 4.6 gives a conclusive statement about the main findings 
of this chapter”. 


4.1 Introduction 


The market-based controller developed in Chapter 3 is able to achieve a solution that is 
technically feasible in the sense that each market-clearing solution provides compliance with all 
technical constraints as well as zero deviation from nominal frequency caused by local supply- 
demand matching. However, the resulting power injections and the generated wholesale and 
retail prices A;,, Ar, are usually not “monetarily feasible” in the sense that the overall (net) 
payments throughout the network balance to zero, i.e. that all revenues of WPs are covered 
by respective payments from RPs. Moreover, despite the global supply-demand matching, 
the prices generated by CCs are in general unable to reflect local scarcity, since geo-spatial 
network effects such as power flow limits cannot be internalized [TBW15]. In particular, the 
methods discussed so far do not prevent from after-market redispatch by the system operator 
due to transmission congestion. Therefore, in this dissertation the free parameter « is used 
to design regional short-term incentives for the activation of additional power generation in 
undersupplied locations and the (market-based) cutback of power generation in oversupplied 
locations. 


The aim of balancing the overall payments throughout the network can be formalized as 
follows: 


> A preliminary version of the results of this chapter has been published in the journal paper [KZV+ 22]. 
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Definition 4.1 (Balance of Payments (BoP)). Let 


a(t) = | So Re(t)— S55 So Ge) | dt (4.1) 


TEW kEZ iEVZ,k 


denote the time-integrated sum of revenues of WPs and RPs. Then the payments of the 


interconnected power system are said to be balanced if 


lim o(t) = 0, (4.2) 


t— oo 


i.e. if the time-integrated revenues of all WPs are equal to the time-integrated costs of all 
RPs as t > x. 


A weaker requirement is that not the integrated, but only the momentary payments must 
tend towards zero: 


Definition 4.2 (BoP Containment)). The BoP of the interconnected power system is 
said to be contained if the momentary revenues of all WPs are equal to the momentary 
costs of all RPs as t > ov: 


‚im S > Rrt)-—S) >, f(t) =0. (4.3) 


TEW keZieVz,r 


Trivially, Definition 4.2 is fulfilled, if Definition 4.1 is fulfilled, but not vice versa. 


Remark 4.3. If BoP is satisfied, then it immediately follows that the only net (i.e. external) 
payments by network participants are given by the sum of WPs’ cost for power generation. % 


Remark 4.4. If BoP containment is satisfied, then the BoP error is constant with respect to 
time. © 


While real-time zonal pricing is considered to reduce structural and regional mismatches in 
generation and consumption by signaling local grid conditions and thus providing investment 
signals for WPs to deploy generation capacity in load centers [Sta19], it should be noted that 
RPs in particular are also inflexible with regard to geographical displacements of their own 
power production or consumption. Therefore, as a third measure towards fairness among 
network participants, we impose the following normative requirement with regard to the 
retail prices Ar x: 


54 Namely, forcing a hypothetical “competition on locations” among RPs would not only be meaningless, but also 


arbitrarily discriminating. 
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BoP 

Containment kK’ (t) 

(Subsection 
4.4.1) 


Dynamic K” (t) 
Pricing > 
(Section 4.3) 


BoP 

Restoration e*(t) 

(Subsection 
4.4.2) 


Figure 4.1: Schematic diagram of overall procedure for dynamic pricing. 


Claim 4.5. RPs should pay a uniform retail price Ag per amount of power, independent of 


in which price zone they are located. 


As a consequence of Claim 4.5, we choose the uniform retail price as Ar = A (as opposed to 
the cell-specific wholesale prices Ay, = Kz, - A, cf. Lemma 3.48.) 


4.2 Overall Control Structure 


Fig. 4.1 shows the overall procedure towards a purposeful and economically feasible x as 
addressed in the following sections. At first, several approaches for the automatic generation 
of spatially different participation factors K” (dynamic pricing) are proposed in Section 4.3. 
Main focus is given to the development of a real-time congestion management strategy, 
which is derived and elaborated in detail in Section 4.3.1. Subsequently, Section 4.4 discusses 
the generation of an “adjusted” participation factor K, which satisfies BoP. Inspired by the 
terminology used in frequency control (cf. footnote 23), we distinguish in the following 
between a “primary” controller for BoP containment, which has the goal of limiting the 
BoP error to a finite value (cf. Remark 4.4), and a “secondary/tertiary” controller for BoP 
restoration, which has the goal of optimally restoring the BoP error to zero, in order to also 
comply with (4.1). Hence, in Subsection 4.4.1, a regularized x’ is obtained, which already 
satisfies BoP containment (4.3), but not (4.1). Ultimately, in Subsection 4.4.2 the constant BoP 
error (cf. Remark 4.4) is compensated by a suitable restoration signal e“. All methodologies 
necessary to resolve this last component are then derived in Chapter 5. In accordance with 
our main principle adopted so far, all of the components considered in this chapter aim for a 
distributed communication structure. Fig. 4.2 provides an aggregate overview of the controller 
components to be developed in Sections 4.3 and 4.4 as well as their relation to the distributed 
frequency and voltage controller from Chapter 3. We can see a clear resemblance between the 
light gray colored controller components in Fig. 4.2 and the three layers in Fig. 2.3. 
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Figure 4.2: Outline of overall controller structure for real-time zonal pricing. 


A throughout intent of the resulting overall control mechanism is to exercise and implement 
different notions of fairness among network participants: For the interaction between CCs, 
fairness means that regional price differences (imposed via non-uniform entries in K”) are 
supposed to help reward power generation at locations that are favorable from a network 
perspective and to incentivize grid-supportive behavior in both short and long term. At 
individual cell level, fairness is achieved by using market balancing to prevent accumulations 
of positive or negative amounts of capital at any of the CCs. 


4.3 Dynamic Locational Pricing in Real-Time 


Section 3.5 has revealed that each strictly positive K” is economically efficient in the sense of 
Theorem 3.54. By adjusting k” appropriately, specific desired relationships between zonal 
prices are imposed, which can serve as a real-time control mechanism for the reallocation of 
electricity supply from one cell to another. 


In the following Subsection 4.3.1, we apply a feedback control strategy for K” to obtain 
continuous-time congestion management, which is conducted among neighboring CCs by 
adjusting the cell-specific prices. As discussed in the literature review in Subsection 2.2.3 (see 
also Table 2.2), current price-based congestion management techniques are either based on an 
offline calculation of linear sensitivity factors or rely on simplified, network-reduced power 
system models. By contrast, the presented control scheme puts special emphasis on deploying 
a network-preserving model (cf. footnote 27) of inter-cell interaction that considers both 
active and reactive power flows and accounts for the potential overloads on each individual 
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boundary line. In order to further extend this principle to the nodal voltages, Subsection 4.3.2 
introduces an additional feedback control strategy for adjusting the voltage setpoints US* 
and UŠ. Finally, Subsection 4.3.3 discusses various approaches for influencing K” in the 


sense of Contribution 4 in addition to congestion management. 


4.3.1 Real-Time Congestion Management with Zonal Pricing 


We describe the cell topology by the condensed graph G, (as defined in Section 3.1). Let 
Ê; and Qij denote the sending-end active and reactive power flows respectively from node 
i € Vz, to node j € V,, with k # l. As it generally holds that P;; A —Pji and Qi; # - Qi, 


we define 


Ê i Pj, |Pyil 2 [Piil 4 Qj, [Qyil 2 Qil, 


‘ Qm = : 
—P;ji, otherwise, —Q;i, otherwise, 


(4.4) 


and Sn = Pa + Om for each line m € Én All complex power flows Sn are concatenated 


in the vector $ = col „Ist Sim}. With S”°* specifying the maximum permissible 


me{l,.. 
apparent power flow, the congestion factor Cm € C of line m € Ep is thus defined as 


Syn ; 
gmax’ me Ep- (4.5) 


Cm = 


Definition 4.6 (Line Congestion). Linem € Ê is said to be congested, if 


ICm| > 1. 


For the sake of argument, the next example paves the way to a resonable choice of K” in case 
of line congestion. 


Example 5 (Congestion Management by Zonal Pricing) 


Consider a WoC scenario with two cells interconnected by a single power line as depicted in 
Fig. 4.3a. 


Sa 
Bi 


(a) Network with two cells (b) Network with three cells 


Figure 4.3: Example networks to illustrate the influence of K” in congestion management. 
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max 


We assume that the power line is congested (i.e. [Sisal > 5735) with Pra > 0. Obviously, 
the active power flow from cell 1 to cell 2 can be mitigated by raising K3 (to stimulate active 
power injection in cell 1) and/or by lowering KY (to restrain active power injection in cell 2). 
Conversely, if Pı 2 < 0, then raising kK] or lowering K% is advisable. 


Now consider a scenario of three interconnected cells as depicted i in Fig. 4.3b and assume again 
that the power line from cell 1 to cell 2 is ee with [Siml > max and Êi, > 0. 
While the qualitative statements on modifying «k and K} stay the same as in the two-cell 
scenario, it is not obvious whether increasing or decreasing x) is more favorable in order to 
support the congestion management. Indeed, an increased power flow through parallel lines 
1 — 3 and 3 — 2 can contribute to further relieving congestion on line 1 — 2, hence it is 
not reasonable to keep k completely unchanged. This fact applies even more to scenarios 
with several parallel branches or more than one adjacent cells whereby a purely heuristic 


determination of appropriate ki, certainly becomes increasingly complex. 


Apparently, if |S3_,,| = 
3 and 1 and between cells 3 and 2, then an increased or decreased power injection in cell 
3 (triggered by a changed participation factor K3 ) has no effect on the power flow over the 


congested line 1 — 2. On the contrary, i 
the congested line is quite similar to the influence of cell 1, thus cells 1 and 3 should have 


similar participation factors. Analogously, i 
highly similar to the influence of cell 2. Thus, the participation factors xy and K3 should be 
similar. In summary, it emerges that if two cells exchange a high amount of power, then the 
respective participation factors should be tighly coupled. Conversely, the participation factors 
of two cells exchanging little power should be decoupled. 


In order to keep all of the following participation factors limited to strictly positive values in 
Rso, we apply the transformation &; := In(K/’). Then, based on the above discussion, the 
(transformed) participation factor of cell 3 can be calculated out of pı and dz as the weighted 
sum 


|S3s1ld1 + [832162 
53-1 | T |S3_,9| 


d3 = (4.7) 


Following the same principle, the participation factors for cells 1 and 2 are calculated as 


|So-41|61 + |So+3|¢3 — %2 
[S21] + [S23] 


\S12l62 + |Si+3l¢3 — Yı 
|Si_,2| + |Sı=3]| 


gi = ; p2 = , (4.8) 


where yı and w are suitable activation functions (yet to be determined) which indicate the 
congestion status of the boundary lines incident to the respective cell. In particular, choosing 
pı > 0 signals that increased power injection in cell 1 has an undesirable impact on congestion 
of the incident line 1 — 2 (i.e. the participation factor of cell 1 should be reduced), and p2 < 0 
signals that increased power injection in cell 2 has a desirable impact on congestion of the 
incident line 1 — 2, thus the participation factor of cell 2 should be increased. 
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Combining (4.7) and (4.8) finally yields 


Si], lil Sis 
y= [S12 —|Sial — 5223| u 1523| _ Q. (4.9) 
|Si>3| [S23] -|51=3]| = |So_-53| 
=A 


It can be seen in (4.9) that A is the Laplacian matrix of the graph in Fig. 4.3b weighted by the 
apparent power flows between cells. 


Based on the insights gained from Example 5, we can thus set up 


0 = — Dpdiag{|5|}D, 6 - u (4.10) 
eS SS 
A 


for the determination of participation factors k” = exp(@). As discussed in Subsection 3.4.1, 
constraint (4.10) can be obtained in feedback form via the gradient flow 


Ts = —Ad — Y. (4.11) 


However, while in Example 5 only line 1 — 2 was assumed to be congested, in a general setup 
potentially all lines can be more or less affected. In order to map the individual contribution 
of each cell to (possible) congestion on one or more lines, we choose the activation functions 


ap to 
p = D U, (4.12) 


where U € Rll = col neg, {Om} is an appropriate barrier function penalizing lines m € Ê 
P 

which are close to congestion. The signal flow diagram in Fig. 4.4 briefly summarizes the 

influence of line congestion on the prices A(t) and thus the active power generation p, (t). 


Before discussing two possible options how to properly select Um, we conclude with three 
additional remarks: 


Remark 4.7. Using the logarithmic quantities & instead of K” also has the practical benefit 
that arithmetic averaging (4.7) or (4.8) clusters around the social optimum with kK” = 1 and 
thus no upward outliers can occur. © 


Remark 4.8. For high voltage networks where it is justified to assume that power lines are 
purely inductive, voltage profiles are flat (i.e. U; ~ 1 fori € V), and voltage angle differences 
Vij are small, power line susceptances can be used to approximate power flows throughout the 
power system [vDD14]. Accordingly, it is then valid to use the approximation 


A= Dydiag{|5|}D,, ~ -BD,diag{B}D,, (4.13) 
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Figure 4.4: Signal flow diagram illustrating the relationship between line congestion C(t) and prices A(t) created 
by the proposed real-time congestion controller. 


where 8 > 0 and B € Ree! is the vector of susceptances of inter-cell power lines (with the same 


sorting as the edges in D,). The approximation in (4.13) has the major advantage that A is 
now independent of the momentary power flows over inter-cell lines and thus can be calculated 


offline. © 


Remark 4.9. An insightful interpretation of (4.10) can be made via generalized coordinates: In 
this regard, pp (and thus ki!) can be interpreted as the potential (i.e. generalized effort) variable 
of cell k, while the apparent power flows Sst, which indicate how closely the corresponding 
values of & should be coupled (cf. Example 5), act as generalized conductances. Accordingly, 
wy can be interpreted as local injection of generalized currents (i.e. as generalized flow variable), 
inducing differences of potential (i.e. generalized efforts)—in particular, an increase of the local 
potential (if wx, > 0) or decrease of local potential (if Yg < 0). Formulating the node-voltage 
method for the resulting generalized circuit with potentials &, conductances IS , and flows wp 
also leads to (4.10), where A acts as nodal admittance matrix. © 


Congestion Controller without Integral Action 


As indicated in (4.10), a valuates the impact of each cell on overall congestion. By using (4.12), 
we can assemble this impact from line-wise “static” barrier functions to increasingly penalize 
those lines that are close to congestion. With regard to Example 5 where |.$;_,2| > SP23 
and Pi,2 = R{Si42} > 0, the participation factor x} has to be decreased in order to 


alleviate congestion between cells 1 and 2, while «4 should be increased. Conversely, if 
Pı 2 = R{ S12} < 0, then «// has to be decreased and x) has to be increased. 


In this light, we propose a static barrier function for each line as follows: 


C Be f 
R CH Re | m m ; CH > Cmm, 
Bm = on} aa e T (Onl > Cm (4.14) 


0, otherwise, 


where C™™ € (0, 1) is a (user-defined) threshold for control actions. As long as |Cm| is below 
this threshold, it holds that Um = 0. Moreover, |Um| — 00 as |Cm| — 1°° (see Fig. 4.5). 


Remark 4.10. Note that the individual values of dr, and pp in (4.11) can be calculated locally 
by the respective CC k € Z, since the CC only requires information about neighboring K values 
as well as power flows across its own cell boundary. © 


55 


Analogously, it can be stated that |Um| — + as |Sim| > Fmax, 
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kin 


0. °8{Cn} 


N 


He} = RfC„} 


Figure 4.5: Exemplary plot of barrier function Um (Cm) with cmin — 0.5 from different viewing angles. The red 
circle highlights all points where |Cm| = CM” holds. 


Remark 4.11. The proposed barrier function (4.14) represents only one possible choice of 
feedback for w = D,v. Basically, the only requirement which Ö must certainly satisfy is 
sen(Om) = sgn(R{Cm }) in order to ensure that ab has the “correct” sign which helps to alleviate 
congestion. Furthermore, |O,,| should be at least monotonically increasing with respect to 


ICml- © 


Congestion Controller with Integral Action 


Typically, the maximum rated power $™* can be exceeded for a certain amount of time 
before a secure operation of the power system can no longer be guaranteed [CVV17]. Thus, 
the momentary apparent power flow should generally be permitted to temporarily exceed 
the maximum rated power. In order to capture this fact, we propose a more sophisticated 
feedback law for a by incorporating a simple model for line overheating to replace the static 
feedback (4.14), namely 


= E Cnt < Cm, 
Om 
Öm = 4-1 = [Emm + (ml = CBIR {Cm}, EM" Cl S1, (4.13) 
= R{Cm}, [Crn] >, 
Om 


Here, cue denotes the threshold below which the line is defined as cooling. Accordingly, 
[Um] can be interpreted as a measure of line overheating. In particular, if the line is congested 
(|Cm| > 1), then it heats up and |U,,| in (4.15) increases. If the line is not congested and 
|Cm| < C™™, then it cools down (such that |O,,| in (4.15) decreases) and the cooling rate 
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increases proportionally with less power flowing over the line. If C™™ < |C„| < 1, then 
both effects are overlaid. The design parameters 7; , Taa > 0 model the specific heating 
and cooling rates, which are dependent on e.g. the conductor type of the power line or local 
weather conditions [Dou88]. 


Both Remark 4.10 and Remark 4.11 analogously apply to the dynamic barrier function in 
(4.15). 


4.3.2 Adjustment of Voltage Setpoints 


If the voltage controller (3.110b)-(3.110c) is equipped with setpoints U", U5!" as proposed 
in Remark 3.45, additional control of the reactive power flows can be implemented as a second 
measure for congestion management. For this purpose, the setpoints U}°' can be used to 
manipulate voltage differences along congested lines and thus mitigate congestion by reducing 
reactive power flows where necessary. 


Reconsider the exemplary scenario from Example 5 with line 1 — 2 assumed to be congested 
with |Si_,2| > SP25 and Qi42 = S{ S152} > 0. Here, the reactive power flow can be 
decreased by increasing the voltage amplitudes in cell 2 and/or by decreasing the voltage 


amplitudes in cell 1, while the voltage amplitudes in cell 3 do not need to be modified”. 
In general, cell-specific setpoints U;** for each k € Z can be controlled by the dynamic 
setpoint correction scheme 


. =A 
Tyee UR = —UR* + UR +U (UR), (4.16) 


—A 
where U}°" denotes the nominal voltage of cell k € Z, U > 0 is the maximum admissible 
deviation between the steady-state voltage setpoint and the nominal voltage, UÂ is the 
“desired” voltage deviation between U fee and UP°™, and 


i, - eS dy 
(US) Up, -1< UP <1, (4.17) 
=1, Uô <1 


. ; as : À A 
is a saturation operator restricting U onto the domain [—1, 1]. With the summand U (Uf)! , 
in (4.16), it is guaranteed that the voltage setpoint U;** always remains in a sufficiently small 


neighborhood around the nominal voltage U7°™. 


A simple yet effective feedback control scheme for the desired voltage deviations U A can 
then be applied by choosing for example 


U= DS) sgn(S{Cm}) + ml. (4.18) 


meee 
m=i> j, i€ Zk 


5 An essential difference between active and reactive power flows is that reactive power flows are mainly local 


phenomena and thus should also be compensated by local interventions [vDD14, LAPG17]. 
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Here, the desired voltage deviation Uf is proportional to the absolute value |C,,| of the 
congestion factor. The multiplier sgn(S{C,,, }) reflects the direction of reactive power flow, 
which is required in order to yield the correct orientation for UÂ. With respect to all cells, we 
thus get the local setpoint correction dynamics 


pala +0" 40^ 


U^ = D, (sgn(S{C}) 0 |C]), (4.19b) 


Me, (4.198) 


: set set en nom 
where Tyse = diag,ez{Tuse} U = colkez{Uk" HandU = colkez {UO}. 


4.3.3 Extensions, Combinations, and Long-Term Perspective 


In Subsection 4.3.1, the participation factor K” was exploited to establish a real-time congestion 
management strategy. In the following subsection, we briefly discuss several other possible 
means for regulating K” such that alternative notions of “grid-supportive behavior” as objected 
in Chapter 1 may be realized. 


A somewhat classical approach is to ponder on the active power balances separately for each 
cell”. By e.g. setting 


(4.20) 


the participation factor is equal to 1 as soon as the active power balance is fulfilled for 
cell k € Z, while the revenues for power generation automatically increase if cell k is a 
load center. By this basic approach, regional incentives can be created to deploy additional 
generation capacity primarily where there is a momentary or permanent local undersupply. 
Conversely, cell-specific prices automatically drop in case there is a lot of available local 
generation capacity and/or few local consumption. Since all information required in (4.20) 
can be measured decentrally in cell k, there is actually no need for communication with 
neighboring CCs. An obvious disadvantage of (4.20) is that there exists no solution for Ki 
whenever there is no momentary power generation in cell k for some reason. 


A considerably simpler approach, which invariably yields a nonsingular solution while requi- 
ring no measurements from WPs’ power generations, is 


>, palt) 


„ iEVZ,k 


ern 


kEZiEVZ,k 


(4.21) 


>7 The concept of cell-specific balancing is closely related to the MG paradigm, which suggests that local power 


balancing helps to reduce (global) network losses, cf. [Sch15, p. 56]. 
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In this case, the pricing scheme only accounts for the momentary distribution of consumption 
among all cells. To obtain meaningful results for this approach, cells should have nominally 
the same amount of consumers. In contrast to (4.20), it is necessary to obtain delay-free 
information about the consumption in all other cells*®. 


Besides the examples shown above, also a combination of pricing regimes can be applied such 
that the controller equations for #7 (t) (and consequently the incentive signaling of the pricing 
strategy) are composed of several components in an additive manner. For the remainder of 
this subsection, denote by «x? the “cell-individual” calculations governed e.g. by (4.20) or (4.21). 
Then, by introducing a communality factor ı € [0,1] and choosing 


Ky(t) =e- K+ (1-1) Kl), K>0, (4.22) 


the degree of excess among cell-specific prices can be seamlessly adjusted from fully indi- 
vidualized prices K (t) = «;(t) for ı = 0 up to a commonly shared price x’ (t) = K for 
t=1. 


Besides a fixed common value K > 0, more sophisticated methods for the “commonly shared 
part” of (4.22) can be employed. The following lemma exemplarily shows how dynamic 
consensus among CCs can be reached by by invoking distributed averaging theory [DdGP10, 
SDF17]. 


Lemma 4.12. Let A, be the adjacency matrix of a weakly connected and possibly 
weighted communication graph 9, = (Z,€,). Then for each equilibrium ((K,)*, er) 
of the distributed-averaging-based controller 


Ky =b-e, +(1—2)- KR, kez, (4.23) 


êk =— 5 @rıler — Ky), keZ (4.23b) 
lez 


with convex weights op = [Ax |xı/ (Siew [A.]xı), it holds that e* is a weighted average 
of the neighboring participation factors ki’. 


Proof. By replacing wm = [A}nu/ (Siew, [Alrı) in (4.23b) and equating the right-hand side 
of (4.23b) to zero, we get 


ek = Alkı (K (4.24) 


lez 


Thus, e% is a weighted average of all neighboring participation factors (x}')*, whereby the 
weights are given by A. 


Remark 4.13. If, is complete, then ej, = ef holds for all k,l € Z, i.e. the commonly shared 
part ofk” (t) in (4.23a) has the same value throughout the whole network. © 
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(J)ı=1 (b) ı = 0.5 with complete G, (c) t = 0.5 with Y,, = 9% (dı=0 


Figure 4.6: Zonal prices in case of load center in cell 7 under different choice of communality factor ı. The intensity 
ofthe red color in each individual cell signals the resulting increase in local prices and thus the added 
incentive for supply in this respective cell. 


Fig. 4.6 schematically shows the effects of combined pricing (4.23) for a load center in cell 7 
for different communality factors ı € {0,0.5,1} with x? (t) being set according to (4.21). It 
emerges that ı = 1 leads to a regionally independent (uniform) price (see Fig. 4.6a), while for 
t = 0 the load surplus in cell 7 leads to an increased price in the same cell only, while all other 
cells remain unaffected (see Fig. 4.6d). For ı € (0, 1), on the other hand, the specific properties 
of the weakly connected graph Y,, become relevant: As Fig. 4.6b (complete graph Y,,) shows, 
all cells other than cell 7 share a unique price A47 < Ar, while in Fig. 4.6c (incomplete graph 
G,, = G.), the prices A, in cells k # 7 gradually drop with their spatial distance to cell 7. 


4.4 Dynamic Balancing of Payments 


While the previous section focused primarily on notions of fairness for WPs, this section is 
devoted to also ensuring fairness for RPs in terms of Claim 4.5 by further refinements on 
participation factors k”. In particular, Subsection 4.4.1 combines the fairness objective of 
Claim 4.5 with the BoP containment from Definition 4.2. Subsequently, Section 4.4.2 introduces 
an optimization-based framework for BoP restoration to also comply with the BoP requirement 
as in Definition 4.1. 


4.4.1 Balance of Payments Containment 


Before considering specific actions for the observance of BoP containment, it is important 
to note the basic finding that the resulting power flows are invariant with regard to scalar 
multiplications of the participation vector K”: 


>8 Even if this approach is not decentralized, a fully distributed consensus-based control scheme can be derived by 


applying Lemma B.1 analogously to the control laws in Section 3.4, such that calculations of (4.21) are solely 
based on neighbor-to-neighbor communication among CCs. 
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Lemma 4.14. Let (p}, U}, UŁ) denote an equilibrium of (3.110a)-(3.110i), (3.147a)- 
(3.147b). For a given kı, and let (pj, Us, U} ) denote the corresponding equilibrium for 


Ky = d+ Ky with a > 0. Then it holds that (pt, U}, UŁ) = (pt, Uy, U%). 


Proof. With Theorem 3.51 it follows that each equilibrium (pi, Ut, UŁ) of (3.110a)-(3.110i), 
(3.147a)-(3.147b) with x = «kı is defined by (3.154) with (3.154a) equal to 


0 = —Ve* (pl) — wt + A al — pl, (4.25) 


while each equilibrium (pz, UF, U?) of (3.110a)-(3.110i), (3.147a)-(3.147b) with x = k2 
fulfills 


K * * \* ax ^x 1 k x * ai ak Ak 
0 = -Vc (p3) —w +A + Âg- — figs = — VE (pi) —w +A + Pg — Mey, 
(4.26) 


a > 0, together with (3.154b), where each (-)* in (3.154b)-(3.154l) is replaced by (-)*. With 


Lemma 3.35, it follows that w = w* = 0. Multiplying (4.26) by a thus leads to 
0=-yV (pt) + A) + at_ — a! 4.27 
(p$) + Hg— Kor ( $ a) 
= -Vo" (pt) +a: Â +a-pt_—a- igy. (4.27b) 


By comparison of coefficients” between (4.27), (3.154b)—(3.154l) and (4.27b), (3.154b)-(3.1541)°, 
it immediately follows that for each equilibrium (-)* to x fulfilling (4.27a), (3.154b)-(3.1541), 


there exists an equilibrium (-)* to k2 = a- kı with Pz = pi, N =a. x fl =a: in; 
Ber = a: fig, HS = BS. OB, = == aL, UE = U$, U4 = UF, 
and d* = D*. In particular, it holds that (pi, Us, Ui) = (pz, Uf, U7). 


Lemma 4.12 implies that there exists a remaining degree of freedom of dimension one which 
only affects the prices (and thus the costs and revenues of WPs and RPs), whilst leaving all 
physical quantities (i.e. power flows and nodal voltages) unchanged. This degree of freedom 
can be exploited to generate a regularized K’ (t) = a* - K” (t), where a* is the unique scalar 
that induces BoP containment without altering the physical actuation (and thus the incentives 
originally governed by K” (t)). With the observation that the requirement of BoP containment 
(4.3) can be written as ((«’)*) "pz = 1" py, the defining condition for a* thus equals 


a) (dt I Din = 0. (4.28) 


keZ kez 


°° Note that strict convexity of C" (p, ) implies that VC" (p, ) is invertible. 
60 Again, each (-)* in (3.154b)—(3.1541) has to be replaced by (-)*. 
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Due to the separability of (4.28) with respect to k”, we again employ the distributed consensus- 
based control approach derived in Appendix B.2 to obtain the distributed controller 


Tad = —a 0 K" op, + py + Dpva, (4.29a) 
aT 
Tv, Da = —DpYa- (4.29b) 


Using (4.29), each individual CC can execute its control actions by relying solely on own local 
measurements as well as on neighbor-to-neighbor communication, thus in the same way as 
already implemented for the congestion controllers in Subsections 4.3.1 and 4.3.2. 


Remark 4.15. It is convenient to resort to the cell incidence matrix D, as the communication 
matrix in (4.29), as it is already encountered for the settlement of cell-specific participation factors 
ky (4.11) via A in (4.10). Hence, existing communication infrastructure can be reused. O 


Remark 4.16. Ifthe distributed controller (4.29) is interpreted as primal-dual saddle-point 
flow (cf. Subsection 3.4.1), then each equilibrium (a*,v*) of (4.29) is an optimizer of the convex 
optimization problem 


Pen 0 
1 = 
min 5 (Re)! Er RK (4.30a) 
kK! x 
Be, 
0 ag 
subject to (K) Dg =1'p, (4.30b) 


where ay = (kK,)*/Kyz and v% is the vector of Lagrange multipliers for the affine constraint 
(4.30b). One possible generalization of (4.30) is to replace (4.30a) by an arbitrary convex objective 
function J” (K). This optimization problem then also leads to participation factors fulfilling BoP 
containment, without the need to compute non-regularized participation factors k”. However, 
since the relation k’ = a: k” with kK” > O has been dropped, the strict positivity requirement is 
obeyed separately, yielding 


min J* (x) (4.31a) 

K'a 

subject to D,a —K' op, + pp =, (4.31b) 
k'—e>0, (4.31c) 


where € > Q is an arbitrarily small safety constant. Using (4.31), the distributed consensus-based 
controller becomes 


Tpk’ = VI (6!) + ULB, + Mes (4.32a) 
7.4 = -D Vn, (4.32b) 
Toby = Dpa — K' op, + Py, (4.32c) 
TR te), (4.32d) 
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where v,, and u „ denote the Lagrange multipliers for constraints (4.31b) and (4.31c), respectively. 


© 
From (4.28) in conjunction with (4.29), it immediately follows that a* = a* : 1, where 
—1 A —1 
dL 2 (kopen) I Dd. le) Jo 2o (wk Oe) 
Are kEeZieVz,r n kEZ iEVZ,k i kEZ iEVZz,k 
XO pei XO pei XO pei 

iEV iEeV iEV 

(4.33) 


Equation (4.33) reveals some interesting connections between the steady-state wholesale 
prices Ax = «x : (AP)* = a* - (r}/)* - A% and the steady-state retail prices Az = (A°)*, which 
are discussed below. 


Remark 4.17. In the lossless case, the inverse of a* is a weighted average of the individual 
cell-specific participation factors («")*. Conversely, (4.33) suggests that for the case of uniform 
prices k’ = 1, BoP containment is only fulfilled for the (academic) case of lossless networks. In 
other words, the social optimum never produces a (monetarily) balanced solution in real-world 
power systems unless K” is regularized by the scalar multiplication x’ = a* - k” with0 < a* < 1. 
Without this regulation, the monetary imbalance increases as transmission losses increase. < 


Remark 4.18. The regularization by means of a* (respectively a* ) does not only account for 
the aggregate transmission losses™, but also ensures an automatic and fair monetary reallocation 
of these transmission losses to all RPs in real time. Accordingly, since 1' P, > 1' p, for lossy 
networks, there always exists at least one cell k € Z with ki, < 1, regardless of the choice of 


k" € Ra. Through the representation (4.29), each CC is able to calculate the momentary value 
a* in a distributed way. G 


4.4.2 Balance of Payments Restoration 


Recall the time-integrated sum of payments from Definition 4.1 with (3.78) and (3.88): 
t 
o(t) = J — 5 (Ak Kk ` Posi — WR : Pg,k) + 5 A° : Dek dt’. (4.34) 
0 kez kez 


Applying the above “primary” control actions for BoP containment (4.3) leads to a finite, but 
generally nonzero end value o” (cf. Remark 4.4). If o* > 0, then there exists an excess of 


61 Transmission losses are already considered by the automatic dispatch of Pg developed in Chapter 3 and therefore 


do not have to be explicitly contained in (4.33). 
By contrast, the current electricity market always requires additional grid tariffs to rebalance the resulting 
mismatch. See the discussion in Section 2.1. 
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capital since the RPs’ payments are higher than the revenues that have been received by WPs. 
Conversely, if o* < 0, then a deficit of payments exists because the revenues of WPs exceed 
the payments from RPs. In the following, we will derive the “secondary/tertiary” control 
actions to optimally regulate o towards zero as t — X. 


A reasonable choice for the additive restoration signals e“ is given by the first-order lag 
relation 


= + Pek’ O + Up, kez (4.35) 


where u; is a cell-specific control input yet to be determined, and where the positive sign of 
+Pg,r : o ensures that an excess of overall payments by RPs (o > 0) yields a positive correction 
c% > O (and vice versa), 


Now the overall balance (4.34) along with the cell-specific e dynamics (4.35) can be written 
compactly by the port-Hamiltonian representation 


En O 0 Pe N 0 I 0 
A a ( a d u or 0 VHs(&o) + oT) etal (4.36a) 
SY R 

Lo Jo Ro PA 

1 0 K\T_K 1 2 
Be) Se 3 (4.36b) 


where A’ := (1/A°)- I > 0, and d = A°- (119, — Dk’) is a disturbance that decays to 0 as 
t > oo. 


Lemma 4.19. Let the cell-specific power injections p, # 0 be constant and let (4.3) hold. 


Then the autonomous system (4.36) exhibits a globally asymptotically stable equilibrium at 
the origin. 


Proof. Consider the Lyapunov function candidate Ho (£o) > 0 in (4.36b). The time derivative 


of H,(x,) along the trajectories of (4.36) is given by H, = —(e")'e*. Now define the 
set E = {æo € RZ! : H,(£o) = 0} = {a, € RIZI+1 : e“ = 0}. Since the 
largest invariant set contained in E is the singleton {(e" = 0.0 = 0)} and H,(a,) is 


radially unbounded, global asymptotic stability of the origin follows by LaSalle’s invariance 
principle [Kha02, Corollary 4.2]. 


Now the “optimal” control input u = u* shall be chosen in such a way that the resulting 
closed-loop trajectory ((e”)*(t),o*(t)) of (4.36) is regulated towards zero so that a user- 
defined quadratic Lagrange-type performance index is minimized. In particular, we specify 


te 
min f (E) Qe +Q- 0° +u' Su dt (4.37a) 
E 0 
subject to (4.36), (4.37b) 
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where Q, = 0 and Q2 > 0 are weighting factors for the transient error of the local restoration 
signals e”(t) and the overall balance o(t), respectively, and the control weight matrix S > 0 
penalizes the inputs u(t). 


In fact, (4.37) entails an optimal control problem with quadratic performance index. This 
problem class is extensively studied in the following Chapter 5 by deriving explicit feedback 
control laws for the optimal control input u*. Furthermore, the proposed methodology will be 
applicable to general Lagrange-type performance indices (cf. Definition A.7 in Appendix A.2) 
subject to general (nonlinear) PHS dynamics. 


4.5 Analysis of the Closed-Loop System 


This section studies the closed-loop equilibrium resulting from the interconnection between 
the physical plant system (3.49) and frequency and voltage controller (3.110) from Chapter 3 
along with the dynamic pricing scheme from Chapter 4. It concludes with a stability analysis 
of the individual subsystems with respect to this equilibrium. 


Theorem 4.20. Each equilibrium of the closed-loop system (3.49), (3.110), (3.142), (4.29), 
(4.11), (4.15) has the following properties: 


a) For alli € Vg N Vz, with given prices Aj, = Xj, the active power generation p, ; is 
a solution of (3.83). 


b) All frequency deviations are zero, i.e. w* = Q. 
c) For alli € Vz, it holds that U} = De if no voltage limits (3.83c) are active. 


d) BoP containment (4.3) is fulfilled. 


e) Congestion is prevented, i.e. it holds that |S'| < S”””* whenever the integral action 
feedback (4.15) is used for congestion management. 


Proof. a) Since (3.110a)-(3.110i) is the projected saddle-point flow of the convex optimiza- 
tion problem (3.83), it follows from Lemma 3.29 that each equilibrium of (3.110a)-(3.110i) 
with A, = Afis a solution of (3.83). 


b) This statement immediately follows from Lemma 3.35. 


c) For each i € Vs, the equilibrium condition of (3.110g) equals 


UF; = Uti, (UF; = Us i > 0) and (Sri > 0), 


(4.38) 
0, otherwise. 


0= (Ur; x Den)ua,; z i 


Since we assume that no voltage limits are active, it holds that Ur ; < Uri. Thus, (4.38) 
states that ws, ; = 0. The same reasoning can be applied to the steady-state conditions 
(3.110f), (3.110h), and (3.110i) to obtain us— = 0, wz_ = 0, and uz, = O. Using these 
results in conjunction with the equilibrium conditions of (3.142), we get UZ; = (Uy 
for all ¿ € V,, which is equivalent to c). 
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d) Since D, is the incidence matrix of a weakly connected graph, each equilibrium of 
(4.29b) fulfills až = ... = aX =: a*. Left-multiplying (4.29a) with 1' yields 0 = 
—a* ((«/)*)" Pg + 1" py, which is equal to the BoP containment condition (see (4.3)). 


e) For any m € &,, let |S*,| > S™** hold such that e) is violated. Accordingly, |C,.| > 1, 
and (4.15) implies that Ùm = ——R{Cm} 4 0. Hence it follows by contradiction that 
Töm 


each equilibrium of (3.49), (3.110), (3.142), (4.29), (4.11), (4.15) fulfills IS] sr, 


Remark 4.21. Requirement c) in Theorem 4.20 can always be fulfilled by choosing g^ suf- 
ficiently small, which prevents the limits on excitation and inverter output voltage from being 
reached. 


Theorem 4.22. If k” > 0, p, > O and py are constant, then the steady-state (a*, v$) of 


the balancing controller (4.29) is globally asymptotically stable. 


Proof. Let £y := (a, Va) be the state of (4.29), and consider the Lyapunov function candidate 
1 
Ve) = 5 ((a- a*) rala — a*) + (Va = v4)" Ty, (Va — vi). 


Since T,.) > 0, it follows that V (£y) > 0 for all £y A & . The time derivative of V (€y) along 
the trajectories of (4.29) equals 


V(éy) = —(a — a*)' (K" o Py 0 (a — a*)) 


+ (a — a*) Dp (Va — vt) — (va -vi)'D,(a-a‘) (4.39) 
saai aiana 
=0 
= (a—a*)!' -diag(r” o D,)(a — a*). (4.40) 
T: 


Ask” > 0 and D, > 0 hold per definition, the matrix T in (4.40) is negative definite and thus 
V (£u) < 0. We define the set E = {é EW : V(&)=0}={(a,v,) : a=a*}. The 
largest invariant set contained in E is {£4}. As (4.29) is derived using distributed consensus- 
based control and the centralized steady-state variable a* is unique (i.e. the unique solution 
of (4.28)), it follows by Lemma B.1 that € is unique as well. Since the largest invariant set 
contained in E is a singleton and V (£y) is radially unbounded, it thus follows from LaSalle’s 
invariance principle that & is globally asymptotically stable [Kha02, Corollary 4.2]. 


Theorem 4.23. IfU and A are constant, then the equilibrium &* of the zonal pricing 


controller (4.11) is globally asymptotically stable. 
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Proof. Define &’(t) = 7,&(t) and note that Tẹ > 0. From (4.11), it follows that 117,6 = 
16 = 0 for allt > 0 since A = Dydiag{|S|}D, and 1T, = O". This means 17 ¢’'(t) = 
17¢'(0) for all t > 0. Therefore the unique equilibrium of ¢'(t) is 6* = 6" +0 -1 
with 176” = 0 and d! = 1° A(0)/N. Using the Lyapunov function candidate V(¢’) = 
(6 — 7 H — p), we get V($') = —(6' — p™) Ald! — &*) and V(¢') < 0 for 
all Ø 4 &’* due to positive semidefiniteness of A and (®’ — &'*) ¢ ker( A). As V(¢’) is 
radially unbounded, it follows that &'* (and thus &*) is globally asymptotically stable [Kha02, 
Theorem 4.2]. 


Remark 4.24. For lossless power networks with U; = const. and v;; ~ 0, the Laplacian matrix 
can be assumed to be constant (cf. (4.13) in Remark 4.8). Thus, the zonal pricing controller (4.11) 
turns into a distributed formation controller [OSFM07], where the resulting state trajectory 
&(t) is a weighted average of the (time-varying) reference signal given by the inputs w(t), 
cf. [FYL06, Ren07, KvC* 19]. Q 


Under the premise of timescale separation, the stability statements for the underlying fre- 
quency and voltage controller can be upheld. This feature is formalized in the next theorem. 


Theorem 4.25. Let x = col{ £p, Pg, Hg- Hgy: Uf, Ms_, Hs+ UT, Mr_, Uz4} denote 


the state of (3.49), (3.110). Furthermore, let x and 0 be fixed, the Hessian of Hp (£p) be 


positive definite in a neighborhood around x% and let the conditions of Lemmas 3.39 and 3.41 


hold in a neighborhood around &*. Then the closed-loop equilibrium &* of the intercon- 
nection of the physical plant system (3.49) with the frequency controller (3.110a), (3.110d)- 
(3.110e), (3.110j)-(3.110k) and voltage controller (3.110b)-(3.110c), (3.110f)-(3.110i) is sta- 
ble in a neighborhood around x*. 


Proof. (see Appendix C.1) 


Corollary 4.26. Let £ = col{£p, Pg, Me, Hg+: Uf, Ms_, Hs+, UT, Mr_, M4} de- 
note the state of (3.49), (3.110) where (3.49) is lossless, i.e. G = 0. Furthermore, let k and 


AS 


U be fixed and the Hessian of H,(&,) be positive definite in a neighborhood around 


x5. Then the closed-loop equilibrium &* of the interconnection of the physical plant sys- 
tem (3.49) with the frequency controller (3.110a), (3.110d)-(3.110e), (3.110j)-(3.110k) and 
voltage controller (3.110b)-(3.110c), (3.110f)-(3.110i) is stable in a neighborhood around 


x. 


Proof. With G = 0, condition W2(a) < 0 in the proof of Theorem 4.25 is automatically 
fulfilled, since for lossless grids it holds that p = 0 and Rp (£p) = Rp = 0. 
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The requirements for constant parameters stated in Theorem 4.22, Theorem 4.23 and The- 
orem 4.25 clearly indicate the ubiquitous need for sufficient timescale separation between 
the individual subsystems (see Subsection 3.4.1 and references therein). Even if there is no 
quantitative certificate, sufficient timescale separation can be managed conveniently by the 


controller parameters 7 (.). In particular, to meet the requirements of Theorem 4.25 that « and 


a set 
are constant, Tg and Tyset must be chosen sufficiently large compared to the controller 


parameters in (3.110). In order to meet the requirements of Theorem 4.23, T; and ot must 
be larger than Tọ. In order to cope with Theorem 4.22, Ta and T,, must be larger than Tg 
and the other controller parameters in (3.110). Thus, the individual controllers operate on 
three different timescales“ as depicted in Fig. 4.2. We can clearly see the correlation between 
the three components in Fig. 4.2 to Contributions 1, 2, and 4 outlined in Fig. 2.3. In Fig. 4.7, 
the individual dependencies and signal flows between these three components are plotted in 
more detail. A comparison with today’s separation of timescales shown in Fig. 2.2 indicates 
that by unifying all tasks into three sub-tasks, there is a sufficiently large “gap” between the 
individual controller parameters possible—specifically, a factor of at least 100 can be chosen 
between each T..,. For further classification and conclusions, please refer to Chapter 7. 


4.6 Discussion 


This chapter has presented a distributed control scheme for the automatic settlement of 
temporally and spatially differentiated prices. The developed continuous-time pricing scheme 
which is solely based upon neighbor-to-neighbor communication between CCs comprises of 
two conceptually distinct steps: 


In the first step, a dynamic locational pricing scheme is provided by means of an automatic 
control of cell-specific participation factors K% (t). By modulating the uniform price A°(t) 
locally, the resulting pricing strategies are able to account for the current scarcity of (marginal) 
production in a specific location at a specific point in time. In Section 4.3, we have introduced 
different approaches with and without integral action for the practical choice of K” (t). In 
this context, special emphasis has been put on the development of a new real-time approach 
for distributed, flow-based congestion control. Through its “holistic” consideration of both 
active and reactive power flows along with integral action, the congestion controller enables 
the possibility to accurately incorporate (and thus exploit) the thermal load capacity of each 
separate inter-cell transmission line. 


In the second step, regularized zonal prices «/ (t) are generated out of K} (t) such that each 
resulting closed-loop equilibrium satisfies BoP containment. A salient feature of this regular- 
ization mechanism is that the allocation of WPs’ power injections obtained in the first step 
are left unaltered, while at the same time all RPs are provided with a retail price Ag = \* that 
is independent of the individual cell and by which all transmission losses are recompensed in- 
stantaneously. The combination with the optimization-based frequency and voltage controller 


63 Note that these three timescales do not correspond to the classical hierarchy of primary, secondary and tertiary 


control. Instead, the controller (3.110) presented in Sections 3.4 and 3.5 covers all aspects of the classical frequency 
and voltage control, while the above control layers shown in Fig. 4.2 address additional (higher-level) objectives 
as suggested in Contributions 2 and 4. 
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Figure 4.7: Controller structure for real-time zonal pricing with three different timescales. Subsystems that are 
depicted at the top act at slower timescales, those depicted at the bottom act at faster timescales. 


(3.110) deployed in Chapter 3 provides economically efficient equilibria where the adherence 
of network constraints is fulfilled at market clearing without the need of a “global” (superordi- 
nate) authority, after-market settlements, or any other type of side payments. Moreover, due 
to the automatic regulation of wholesale and retail prices, no additional BRPs are necessary 
any longer. 


So far, we have not developed an explicit procedure for determining the restoration signal 
e" (t) mentioned in Section 4.1 and Subsection 4.4.2, which allows the resulting participation 
factors Ky = kl, + Ei} to even fulfill (4.2), so that any residual imbalances of costs and revenues 
that may have occured during transient processes can be settled in an optimal way with regard 
to the optimal control problem (4.37). The methodological tools necessary for this task are 
derived in the next chapter. 


5 Optimal Control of Port-Hamiltonian 
Systems 


This chapter is dedicated to the class of optimal control problems for input-state-output PHSs 
(A.13) with Lagrange-type objective functions (A.8) which is a generalization of the BoP 
restoration problem (4.37). Section 5.1 gives a brief introduction to main research branches 
in the field of PHS control. Section 5.2 presents a learning-based methodology for general 
optimal control problems with input-state-output PHSs. Section 5.3 extends the results from 
Section 5.2 to the multi-player case. The main results of Sections 5.2 and 5.3 give rise to 
Contribution 3. Section 5.4 first revisits the application of the proposed methodology to BoP 
restoration (4.37) in order to deliver the remaining steps of Contribution 2, and then outlines 
worthwhile avenues for further research‘*. 


5.1 Introduction 


PHS modeling has had a major impact with respect to energy-based control design (see. 
e.g. [OvMM01, Kug01, OvME02, DBMS09, vJ14]. It opens a different perspective on viewing 
dynamical systems closer to physics and the fundamental concept of energy, which allowed for 
valuable advancements in science and engineering practice. Due to their specific energy-based 
structure with power-conjugated port variables (u, y) and a Hamiltonian H (æ) representing 
the total stored energy in the system, the concept of energy plays a central role in controller 
design for PHSs. In particular, as the PHS representation has a strong link to passivity 
properties (which is exemplarily seen in Subsection 3.4.3), the vast majority of control methods 
for PHSs rely on passivity-based control (PBC). 


Besides feedback interconnection of passive systems [SJK97, p. 107ff.], a common theme in 
standard PBC is the passivation and asymptotic stabilization (under an additional detectability 
condition) via static state feedback u(x) = —k(a) [SJK97, Chapter 2.4; van17, Chapters 5,7; 
OGC04, OvCA08]. Both approaches are comparable in the sense that the controller parameters 
are not unique, but must be found implicitly by solving matching PDEs, which entails a high 
number of degrees of freedom and, more severely, a high degree of computational complexity. 
In order to reduce this complexity, the degrees of freedom are usually confined at certain 
points by setting a subset of the free parameters to a fixed value, thus leading to sub-categories 
such as algebraic or (non-)parameterized IDA-PBC, see [van17, Chapter 7] for a comprehensive 
survey on PBC methods for PHSs. For the passivity-based state feedback controller design, 
an inverse optimality property can be characterized in both continuous [SJK97, p. 107ff.; 
van17, Theorem 3.5.1] and discrete time [MNC15]. For input-state-output PHSs, it is known 


64 Preliminary versions of the results of this chapter have been published in the journal papers [KJSSH21, KJSMH22]. 
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that a static feedback controller is optimally stabilizing with respect to the specific performance 
index 


J (a(t), u(t)) = a (a(t) + uw" (t)u(t) dt (5.1) 
with 


. 
Ka) =- (EO) Fa) + SRT eye). (52) 


if and only if the open-loop system is output feedback passive with a positive definite storage 
function V (a(t)) and provided that an additional detectability condition is fulfilled, see [van17, 
p. 54ff.] and [SJK97, Theorem 3.30] for a detailed discussion. However, classical, PBC methods 
are in general not designed for optimal control problems in a more practical setup, where 
the “forward” solution to a given, arbitrary optimization problem with a general performance 
index (A.8) is sought. E.g. in (4.37), an optimal restoration signal (€”)*(t) has to be determined 
that minimizes the sum of the state error (weighted by Q, and Q2) and control input error 


(weighted by S). 


In the following section, we consider the general case to (4.37) with an arbitrary Lagrangian 
performance index. This problem class is equal to (A.8) with P = {1}, and thus refered to as 
the single-player case throughout this chapter. 


5.2 Single-Player Case 


This section presents a new continuous-time adaptive feedback controller for the optimal 
control of input-state-output PHSs (A.13) with respect to general Lagrangian performance 
indices (A.8) with P = {1}. The proposed control law implements an online learning proce- 
dure which uses the Hamiltonian H (x) of the system as an initial value function candidate. 
Subsection 5.2.1 formalizes the problem class and briefly discusses related work. Subsection 
5.2.2 presents an analytical solution to a modified optimal control problem. Subsection 5.2.3 pro- 
vides necessary and sufficient conditions for H (x) to serve as initial value function candidate. 
Subsection 5.2.4 develops an adaptation strategy by which the modified optimal controller is 
able to learn the (original) optimal control law. For this adaptive optimal controller, Subsection 
5.2.5 proves the asymptotic stability of the closed-loop equilibrium and gives an illustrative 
example. Subsection 5.2.6 summarizes the main result. 
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5.2.1 Problem Definition and Related Work 


Definition 5.1 (Optimal Control Problem for PHSs). Find the feedback control law 
u(x) as the solution of 


. i T 
min N (¢(a(t)) + (u(t))' S(z)u 
subject to «(t) = (J(a(t)) — Reo) + G(a(t))u(t) (5.3b) 


(t) 


)dt (5.3a) 
( 


with state vector x € R”, input vector u € R”, skew-symmetric interconnection matrix 
J € R”*”, positive semidefinite dissipation matrix R € R”*”, input matrix G € R”"*?, 
positive definite Hamiltonian H(a) : R” — R, positive definite L(x) : R” — R and 
positive definite S(x) € R™*™. 


In the case of linear input-state-output PHSs, the Hamiltonian H(a) is quadratic, which 
allows to calculate an optimal controller by using state-dependent Riccati equations [PZCL11]. 
In [MJMFt 09, GCS09, MFMC* 08], the necessary conditions which follow from Pontryagin’s 
Maximum Principle are used to derive an explicit expression for the optimal feedback controller, 
provided the Hamiltonian of the system is quadratic. The authors in [WHLM18] provide full- 
and reduced-order LOR controllers for linear input-state-output PHSs. Further extensions of 
LQ-optimal control for PHSs are given for stochastic or infinite-dimensional spaces [LW18] 
and boundary control systems [LLL20]. 


While there is a rich theory available for linear input-state-output PHSs, the general solution of 
optimal control problems for nonlinear input-state-output PHSs remains challenging due to the 
necessity of explicitly solving the HJB equation which is a nonlinear PDE and thus hard to solve. 
Recent publications use adaptive dynamic programming (ADP) methods to find an iterative 
solution of the HJB equation. In [ASF*20], a combination of neuro-fuzzy and backstepping 
control is proposed for adaptive control of input-state-output PHSs. A profound overview 
on recent adaptive and learning-based control methods for PHSs can be found in [NLJB16]. 
If the performance index of the optimal control problem has a specific structure and the 
system dynamics are given by a Hamiltonian system with controlled Hamiltonian H (x, u), 
iterative learning control [Fuj03, FHS03] and iterative feedback tuning methods [FK08b] have 
been proposed. For fully actuated mechanical PHSs, the authors in [OFM*20] propose an 
adaptive path-following controller from a training trajectory using Bayesian estimation and 
the authors in [ZDC20] propose a deep learning-based algorithm for optimal estimation of the 
system dynamics. [NLJB14] and [SBNL15] use actor-critic reinforcement learning schemes to 
minimize the error between the resulting closed-loop system and a given desired closed-loop 
system without the need of explicitly solving the matching PDE of the employed passivity- 
based controller. However, these approaches suffer from the dissipation obstacle‘°, such that 
the Hamiltonian of the desired closed-loop system cannot be chosen freely. 


6 The dissipation obstacle [OvMM01] states that the Hamiltonian H (a) can only be shaped for coordinates that 
are not affected by physical damping. This limits the applicability of these controller classes to certain physical 
domains, see e.g. the discussion in [ZOJS15]. 
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For the more general class of continuous-time input-affine nonlinear systems, ADP methods 
[BJ16] are proposed where the optimal value function V* (x) can be found iteratively using a 
weighted sum of basis functions [VL10, JJ13]. However, a proper set of initial weights leading 
to a stabilizing controller has to be found by educated guessing. 


In summary, existing optimal control approaches of input-state-output PHSs hinge on very 
specific sub-classes of performance index, system dynamics, or both. Likewise, ADP methods 
for input-state-output PHSs as well as for the more general class of continuous-time, input- 
affine nonlinear systems are non-constructive in the sense that they require the intransparent 
guessing of initial weights for a stabilizing value function candidate. Hence, to the best of 
the author’s knowledge, there exist no explicit control schemes for dynamic optimal control 
problems with generalized Lagrangian performance index and a general input-state-output 
PHS. 


5.2.2 Modified Optimal Control for Port-Hamiltonian Systems 


The initial step of our controller design is based on a trick originally outlined in [SK00]. By mul- 
tiplying the dynamic constraints (5.3b) with the gradient of a control-Lyapunov function (CLF) 
V(x), we obtain a MOC problem that allows for an analytical solution of an asymptotically 


stabilizing u(a(t)) = —k(ax(t)). 


Definition 5.2 (Control-Lyapunov Function (CLF) [FK08a, p. 46]). Leta = f(x, u), 
x € R”, u € R” with f (On, Om) = On. Then a radially unbounded, positive definite 
function V : R” — R, fulfilling 


Ve Ær: i (5.4) 


is said to be a CLF. 


Remark 5.3. For input-affine nonlinear systems 


a(t) = f(x(t)) + g(a(t))u(t), (5.5) 
condition (5.4) is equivalent to [KA01, p. 641] 


T T 
(=) G(x) =, — (=) f(a) rer? - (5.6) 


It is well known that a CLF ensures the existence of an input u = —k(a) such that the 
closed-loop system is asymptotically stable at the origin®. 


66 The formal statement that the existence of a CLF is necessary and sufficient for stabilizability of a nonlinear 


system is given by Artstein’s Theorem [Art83]. A first explicit control law that applies this finding in practice 
was provided by Sontag’s universal formula for stabilization [Son89]. 
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In particular, provided that a suitable CLF is given, it was shown in [SK00] that for general 
input-affine nonlinear systems (5.5), the following MOC problem can be solved explicitly: 


Definition 5.4 (Modified Optimal Control (MOC) Problem for PHSs). 


min | (L(x) + u'S(z)u) dt (5.72) 


U 


subject to V(x) = ea (Jæ) - Rl) =) + G(æ)u), (5.7b) 


Lemma 5.5 (Modified Optimal Control Law). Let V(a) be a CLF of (5.5). Then an 
exact solution of the MOC problem (5.7) is given by the continuous control law 


-S (a)GT (a)? tla), Sr £0, 
O, Sy = 0, 


(a) = fr+ VJ (fr)? + Qr - Sr, 


= Yr 
fela) = (SE) Fle), 


z 
Sy(x): (=) G(x) S~t (x)G! (æ) 
x(x) := (ax). 


Qr( 


Proof. The proof follows the lines of [SK00, Theorem 4.4]. For optimization problem (5.7), we 
get the Hamiltonian 


ef 
H (x,u, T) =5¢(e) = Su’ S(a)u +Y. (&) (f(x) + G(a)u) (5.13) 


with scalar Lagrange multiplier Y. Application of the control equation ar = 0 leads to (5.8). 
As stated in [SK00, Theorem 4.3], a free final time and fixed final state in (5.7a) implies that for 
each optimal trajectory (a*, T*) it has to hold that 5 (x*, u*, Y*) = 0, which is equivalent 
to 


: 
sr (SE) CEST) e" 


4 
-Y* (=) (x*) f(a*) — sna) =0. (5.14) 


116 5 Optimal Control of Port-Hamiltonian Systems 


Since (5.14) is a quadratic function in Y, it has the explicit solution 


a 
fr a Qr: 2 (Z) G(x) #0, (5.15a) 
an (Z) G(x) =0 (5.15b) 


with fr, Qr, Sy as in (5.10)-(5.12). Note that the “+” solution in (5.15a) implies T(a) > 0, 


thus the “—” solution in (5.15a) is discarded (cf. (5.9)) ae, it implies Y (x) < 0, which always 
leads to an unstable solution. Moreover, note that a G(x) = 0 implies fr < 0, since 
V (a) is a CLF. It can be shown by de L’Hospital’s rule [Sac01, pp. 88, 186] that the Lagrange 
multiplier Y (a) in (5.15) is continuous even for the case (arr. G(a) = 0. Hence, it can be 
fully described with the compact notation (5.9). 


5.2.3 Control-Lyapunov Functions for Port-Hamiltonian Systems 


In the following, we investigate under which conditions the Hamiltonian H (a) is a CLF. 


Lemma 5.6. Consider an input-state-output PHS as in (A.13). Then H(x) is a CLF for 
(A.13), if and only if 


(5.16) 


Proof. Since J (æ) is skew-symmetric and H (a) is a CLF, it holds by definition for all x 4 0 
(cf. (5.4)) that 


inf {H(«)} 
=int | (Z HY" (2) - Ra) E + G(e)u)} 

sf- CE) mit (E) a 

<0. (5.17) 


If Ga G(x) # 0, then there always exists an input u’ such that (aay G(z)u’ < 0 is 
fulfilled. If (22) G(a) = 0, then the first term inside the brackets in (5.17) needs to be 


negative whenever x # 0, i.e. 


T T 
(£) G(x)=0 > -(&) Ra (5.18) 
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which is equivalent to (5.16). Conversely, if (5.16) is satisfied, then the definition of a CLF is 


automatically fulfilled since Xg is the space where (ay? G(x) = 0 in Definition 5.26 and 
hence (5.16) is equivalent to (5.6). 


Corollary 5.7. Consider an input-state-output PHS as in (A.13). Then H(x) is a CLF for 


(A.13), if and only if (A.13) is zero-state detectable. 


Proof. Since the input-state-output PHS (5.3b) is equipped with the passive output (A.13b), 
it holds that Ve = {a € R” : y(x) = 0} and thus condition (5.16) is identical with 
the definition of a zero-state detectable input-affine nonlinear system (5.5) given in [van17, 


p. 47]. 


Corollary 5.8. Consider an input-state-output PHS as in (A.13). Then H(x) is a CLF, if 


rank(R(a)) = n holds for all x € R”. 


Proof. Trivially, if R(x) has full rank, then Ka 


fulfilled for all x € IR”, which also implies for all x € Xa. 


R(a) 32 is positive and thus (5.16) is 


Next, we introduce a necessary and sufficient condition under which the Hamiltonian of a 
linear input-state-output PHS is a CLF: 


Lemma 5.9. Consider the linear input-state-output PHS dynamics 


x =(J—R) a + G(a)u (5.19) 


with J =—J', R > 0, H(a) = !x'Qx and Q > 0. Then H(x) is a CLE if and only if 


mo) 


ker{G'Q} N ker{Q' RQ} = 0. (5.20) 


Proof. For linear input-state-output PHSs, it holds that 2H = Qa and thus Ya = {x € R” : 
GH = 0} is equal to ker{G' Q}, since 


13H 


Va €ker{G'Q}: en G'Qx=0. (5.21) 
Consequently, (5.16) in Lemma 5.6 reads as follows: 
OH\' „OH 
T : = T 
Va E€ ker{G Q}: (£) R dr (Qr) RQx<0. (5.22) 


First, it is important to note that —(Qa)' RQz in (5.22) is always < 0 for x € R”, since 
the product Q' RQ of positive semidefinite matrices is again positive semidefinite [HJ12, 
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p- 431]. Secondly, since 2 Mæ = 0 => x € ker{M} for each M >= 0 (cf. [HJ12, 
Observation 7.1.6]), it follows that the equality z'Q'RQzx = 0 holds if and only if x € 
ker{Q' RQ}. Consequently, (5.22) holds if and only if ker{G'Q} and ker{Q' RQ} are 


disjoint. 


Taking Lemma 5.6 into account, MOC can be applied to input-state-output PHSs in a straight- 
forward manner‘’. However, as a consequence of the modification, the MOC law is optimal to 
an unintentionally modified objective function. The next lemma further evaluates to what 
extent the controller (5.8)-(5.12) is also optimal with respect to the original optimal control 
problem (5.3). 


Lemma 5.10. Consider an input-state-output PHS as in (A.13). The (modified optimal) 
controller (5.8)-(5.12) is optimal with respect to (5.3), if 


Ve eR": T(x) =1. (5.23) 


Proof. If Sy # 0, then it follows from (5.11) that Gia # 0. Thus (5.23) is equivalent to 
Sy — 2fr = Qr, and substitution of (5.10)-(5.12) leads to 


T T 
Be) (=) G(æ) S- (æ)GT + (=) fle)=0, 629 
2 2 \ 0x 
which is equivalent to the HJB equation for time-invariant systems. The function V(x) 
solving (5.24) is the value function V*(a). This means, if a CLF is found for which T(x) = 1 
holds for all a € IR”, then this CLF is equal to the value function V* and thus (5.8) is an 
optimal control input for (5.3). If Sy = 0, then Gl (a) = Q. Thus, (5.23) is equivalent to 


T(x) = ayes E 1, which also leads to Sy — 2 fr = Qr. 


A less restrictive requirement can be derived by allowing Y (x) to have an arbitrary but fixed 
positive value: 


Corollary 5.11. The (modified optimal) controller (5.8)-(5.12) is optimal with respect to 
6.3), if 


Va €R”: T(x) =c, ce Ryo. (5.25) 


In this case, the value function V* (a) is ac-multiple of the CLFV (x), i.e. V* (x) = c- V (x). 


Proof. (trivial) 


67 For a special subclass of input-state-output PHSs, it can even be shown that apart from H (x), there exist no 
other viable CLF candidates (see Lemma D.1 in Appendix D.1). 
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Remark 5.12. Lemma 5.10 states that if Y (x) # 1, then the chosen CLF V (æ) does not solve 
the HJB equation (5.24) and is thus not equivalent to the value function of (5.3). Consequently, 
the resulting controller (5.8)-(5.12) is not optimal with respect to (5.3) unless Y(x) = 1. DO 


Remark 5.13. From Corollary 5.11 we can conclude that unless T(x) converges to a constant 
value and remains constant even after a disturbance, the chosen CLF cannot be equivalent to the 
value function. Consequently the resulting controller (5.8)-(5.12) is not optimal with respect to 
(5.3). The stationarity of Y (t) over time can therefore be interpreted as an indicator of fitness 
with respect to the original problem (5.3). © 


Example 6 (DC Motor) 
Consider the linear model of a DC motor from [vJ14, p. 29]. 


t= ® =| one) + fol u, (5.26a) 
OH (a) 
[1 9] Da (5.26b) 


where H(x) = 1/2(x? + x2), with u being the input voltage, y being the output current, 
b > 0 being the mechanical damping constant, R > 0 being the resistance, and K > 0 
being the gyrator constant. Now choose b = R = K = 1 and consider the MOC problem 
(5.7) with L(x) = x? + x} and S(x) = 1 subject to (5.26). By numerically solving the 
algebraic Riccati equation, we can calculate the value function as V*(x) = 1/2”'Pax ~ 
0.2127 +0.02212%2 + 0.2422. 
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Fig. 5.1 shows the resulting trajectories of Y(t) depending on different chosen CLFs V (x) in 


(5.7): 


0 5 10 15 
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5 


10 
tins 


15 


(b) Va(x) = 0.252? + 0.522 


0 5 10 15 
tins 


(c) V3(a) = 0.527 + 0.2523 


5 


10 
tins 


15 


(d) Va(æ) = V* (æ) = 0.212? + 0.022122 + 0.2423 


Figure 5.1: Trajectory of Y for different CLFs V (æ). 


It can be seen that as V (æ) is closer to the value function V*(a), Y is closer to 1. Since V4(a) 
is equal to V* (a), T in Fig. 5.1d is equal to 1. 


5.2.4 Adaptive Optimal Control for Port-Hamiltonian Systems 


Even if H(a) qualifies as a CLF as described in Subsection 5.2.3, choosing V(x) = H(a) 
will generally result in T(a) # 1. With Lemma 5.10, this implies that the HJB equation 
(5.24) is not fulfilled and thus the resulting controller (5.8)-(5.12) is not optimal with respect 
to the original problem (5.3). For this purpose, in this subsection, we propose an extended 
CLF V (a, w) as a linear combination of H(a) and a weighted sum of basis functions. Later 
in this subsection, we devise a gradient-based adaptation strategy of the weighting factors 
w such that V (x, w) fulfills the condition of Lemma 5.10, and present an adaptive optimal 
controller for (5.3) based on V (a, w). In the following Subsection 5.2.5, we then show that 
the equilibrium of the closed-loop system is (asymptotically) stable. 


Extended CLF 


Definition 5.14 (Extended CLF). Let =: R” — R” be a vector of basis functions. Then 
V : R” x R” > R with 


V(x, w) = H(£) + w' 


weR” (5.27) 


is said to be an extended CLF. 
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Assumption 5.15. The basis functions Z(x) in (5.27) are chosen such that there exists a 


vector w* of optimal weights that allows the actual value function to be parameterized as 
V*(x) = H(x) + (w*)'S (2). 


Remark 5.16. With the Weierstraß approximation theorem, Assumption 5.15 is admissible if 
the number of basis functions is large (see e.g. [VL10, p. 881; LLW14, p. 1020f.]). This fact will 
allow us to characterize the deviation from the optimal solution by the distance between w(t) 


and w*. © 


With V(x, w) as in (5.27), we obtain 


OV(z,w) OH pi dE\' 
Ox Ox 


w, (5.28) 


and accordingly the MOC law (5.8) takes the form 


av 

ze -S '(x)G' (x) TR w), Si #0, (5.29) 
0, Sy = 0, 
where 
Tæ, w) = fet WP + Or (5.30) 
St 
j av\' ƏH [ov\' 

ra= (Z) (F@)- Re) Ze - (FZ) © (631) 

avy av 
Si (æ) := x) P (5.32) 
Qy(x) := lm), (5.33) 
K (x) := G(a)S~'(x)G' (a). (5.34) 


Employing the same reasoning as in Lemma 5.10, we will study how to check whether a given 
CLF (5.27) is equivalent to the value function V* in more detail. 


Lemma 5.17. Let V(x) = H(x) + (w°) E(x) be a given CLF with w? € R”. Then 
V (a) is equivalent to the value function V*(a) of (5.3), if and only if 


VaeR”: (x2, w°) € O(a, w), (5.35) 
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where 
O(x.w) = {(x,w) € R” x R” : w' A(x)w +a'(x)w +a(æ) =0}, (5.36) 
= =\T 
A(x) = "E kla) (=) ; (5.37) 


Proof. Let w* € R” be the optimal weighting vector. According to Lemma 5.10, it thus holds 
that T(x, w*) = 1 for all x € R”. As shown in the proof of Lemma 5.10, this condition is 


equivalent to S4 — 2f4,-— Qt = 0. Substitution of (5.30)-(5.34) yields 


ow Era (E) w +2( (E) ro (E) 
(E) uw- r (E) ur 


+ (24) ko _2 ei (I) — Ra) Ë ae). 640) 


With Q(x, w) as in (5.36), condition (5.40) can be written as 
VaeR”: (a, w*) € Q(x, w). (5.41) 


Since the given CLF V (x) is equal to V*(a) if and only if w? = w*, this is equivalent to 
(5.35). 


Adaptation of the Extended Control-Lyapunov Function 


For each fixed x € R”, the set O(a, w) in (5.36) is a quadric. The basic idea of the following 
approach relies on the fact that, according to Lemma 5.17, the optimal weighting vector 
w* is contained in each quadric, ie. (w*)' A(z)w* + a'(z)w* + a(x) = 0 holds for 
allx € R”. Thus for each arbitrary but fixed « € R”, w* can be characterized as the 
minimizer of an objective function JL (æ, w) := (wT A(a)w +a" (ax) w+ a(x))?. Moreover, 
with w* = arg min,, LIE, w)} and due to the fact that w* is contained in each quadric 
O(a, w), it follows that 


{w*} < N arg min { JS (x, w)}. (5.42) 


zeRr 
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If the basis functions are linearly independent, then the intersection of all quadrics is a 
singleton and thus 


{w*} = N arg min {T,(@, w)} . (5.43) 


xeR” 


Since 7° is continuously differentiable, this allows for an adaptation of w(t) by a simple 
gradient flow 
: OTe (x, w 
w=-—a: en w) (5.44) 
Ow 

where a > 0 defines the learning rate. However, note that 7 is in general not strictly convex 
around w*, which hampers convergence of (5.44) towards w* and thus necessitates additional 
conditions for a sufficient exploration of the state space. To circumvent these requirements, 
which are often very hard to evaluate, we formulate an extended objective function Jy (x, w) 
providing strict convexity with respect to w in a neighborhood of w* as follows: 


Definition 5.18 (Extended Objective Function). Let 


Q(x, w) = w' A(x)w + a! (x)w + a(x) (5.45) 


with A(x), a(x), a(x) as in (5.37)-(5.39) be the corresponding quadratic function to the 
quadric O(a, w) in (5.36). Then with cı,...,c, € R”, the function 


Fale, w) = IlL +61, w) +--+ Tela + er, w) 
= (Q(x + c1, w))? +--+ + (Q(a +c,,w))” (5.46) 


composed by a linear combination of shifted objective functions J? (x, w) is said to be an 
extended objective function. 


Lemma 5.19. Let 7,,(a, w) be an extended objective function as in (5.46). Then Jy,(x, w) 
is (locally) strictly convex in an open neighborhood M of the optimal weights w* for all 
x € R”, if and only if the vectors 


vı = 2A(x@ + c,)w* + a(x + c) (5.47) 


withl=1,...,r are linearly independent. 


Proof. Without loss of generality, we choose cı = 0. As strict convexity with respect to w 
needs to be shown, the Hessian of TL and Jw (see(5.46)) are studied. The Hessian of JS can 
be written as 


Selg) = 2(2A(x)w + a(x) (2A(a)w + a(x)" 


+4A(x) (w! A(x)w + a(x)w + a(æ)). (5.48) 
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Since the optimal weights w* need to be part of each quadric O(a, w) regardless of the state x 
(see Lemma 5.17), the second summand in (5.48) is equal to zero for w = w* and accordingly 


Isle, w) 


Ae = wv! (5.49) 


|. 
with vı = 2A(z)w* + a(a). Since the Hessian (5.49) is only composed by the multiplication 
of two vectors, which yields a matrix with identical but scaled row vectors, it is positive 
semidefinite and has rank one. 


For the Hessian of the shifted objective function 70 (a + c2, w), we obtain in a similar manner 
PIL (x + c2, w) 
ðw? w=w* 


with vo = 2A(@%+c2)w* +a(x% +2). Note that the rank of the matrix (5.50) is one regardless 
of the shifting c2. The linear combination J} (ax, w) := J} (x, w) + JL (æ + c2, w) leads to 
the Hessian 


= 2vovs (5.50) 


Ok 


Ow? 


= wv! + wv]. (5.51) 


w=w* 

The same applies to linear combinations with more than two summands due to the linearity 
property of differentiation. It can be seen that (5.51) has a maximum rank of two. As strict 
convexity of Jw is required, full rank r needs to be satisfied for the Hessian of 7,(x, w) at 
w = w*. Thus the question arises, in which case the increase of summands implies a rank 
increase of the Hessian. Each matrix in (5.49) or (5.50) describes a linear map R” — R” of 
rank one and its image is a subspace of R” of dimension one. If the image of > in (5.51) 
is of dimension two, the rank automatically increases, since dim(im(M)) = rank( M) for 
an arbitrary matrix M. With the dimension formula for the sum of subspaces [Bos14, p. 47], 
it follows that dim(Uı + U2) = dim(U1) + dim(%/2) — dim(Uı N U2) for two arbitrary 
subspaces U, and Us. By setting U, = im(v1v1') and U2 = im(v2Vv2") it follows that only 
if dim(Uı N U2) = 0, the summation of the two matrices vıvı! and vvə!" leads to a rank 
increase. For each! € {1,...,r}, the rank-one matrix vv! = (vw, V,+ , v1, vi)" is 


formed by weighted rows of vı! with the respective components v,,i = 1,..., 7 of vi, 
vn, v! 
T_ B 
vvı = : ; (5.52) 
U, vi! 


and hence its image spans the subspace U; = im(vvı!) = {u - vı | u € R}. With regard 
to J} in (5.51) it can be seen that in order to let both sets U; and Ua be disjunct, the linear 
independence of both vectors vı and v2 is necessary. Graphically, the subspace im(v;v; ' ) is 
a straight line in R”, and linear independence leads to non-coinciding straight lines such that 
dim(U, NU2) = 0. Applied to Jw it becomes clear that Jw has a Hessian with full rank, if the 
vectors v; in (5.47) with l = 1,...,r are linearly independent. Hence, each vector v; induces 


d 
a matrix, implying an increase of one for the rank of the Hessian De , which leads to full 
2 
rank and thus positive definiteness of Sr . Moreover, since Jy is a C? function, positive 


definiteness of the Hessian is preserved for all w € M around w*. 
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Remark 5.20. Lemma 5.19 states that the minimum number of summands in Jw to achieve 
strict convexity is r. From a practical point of view, however, it is preferable to compose Jw from 
more than r summands in order to enhance convergence properties. © 


Due to the fact that Q(x, w*) = 0 is fulfilled for all x € R”, the optimal weighing factor w* 
is characterized by the strictly convex optimization problem w* = arg miną {Jw(£, w)}. 
Thus, a given weighting vector w can be adapted by the gradient flow 


. OS lB; w) 
Ow °° 


w = -a a> 0. (5.53) 


Example 6 (DC Motor) - cont’d 
Consider again the DC motor model (5.26) and choose the extended CLF 


V(z,w)=H(z)+[wı we ws] {aire}. (5.54) 


Fig. 5.2 shows the contour plot of the resulting objective function I (x, w) for fixed w3 = w 
and some states: 


(a) zı = —1, z2 = 1 (b) xı = 0,22 =1 (c) xı = 1.5, z2 = 0 
Figure 5.2: Contour plot of JE (æ, w) with w3 = wi. 
The yellow dot marks the optimal point (w¥, w3)" in each case. It can be seen that for each of 
the states in Figs. 5.2a-c, the optimal point is a minimizer of JE (x, w). However, J? (x, w) 
has no strict minimizer for any of the states in Fig. 5.2a—c, which indicates that J}? (x, w) is 
not strictly convex around w*. 
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Now consider the extended objective function Jw(x, w) with shifts co = (0,0)', c1 = 


(—0.5, —0.5)", c2 = (0,1.5)', and c3 = (1.5,0)'. Fig. 5.3 shows the contour plot of 
Fu (x, w) for fixed w3 = w3 and some states: 


(b) xı =0,22 =1 (c) zı = 1.5, x2 =0 
Figure 5.3: Contour plot of Jw(a, w) with w3 = w3. 
In Figs. 5.3a—c, the contour lines around (wł, w3) are closed, indicating that Jw(x, w) is 


strictly convex around w*. 


Now the adaptation procedure (5.53) is applied with both II (x, w) and Jw(x, w). Fig. 5.4 
shows the resulting trajectories of w and T. 


l l l 7 
0 5 10 15 20 
tins 


a) Adaptation usin; JL z, w 
p 8 Uw 


(b) Adaptation using Tw (a, w) 
Figure 5.4: Adaptation results of the weights w depending on the chosen objective function. 
While in Fig. 5.4a, w(t) does not converge to the optimal weights, in Fig. 5.4b w* is reached 


after a learning phase of 3s. Correspondingly, the Lagrange multiplier T(t) in Fig. 5.4a is 
instationary, while in Fig. 5.4b it converges to 1 after having completed the learning phase. 
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Fig. 5.5 shows the trajectories of w(t) and Y(t) for the original objective function I! (x, w), 
if an additive voltage source uq(t) = sin(t/2) serves as an excitation signal of the DC motor 
(5.26). 


Figure 5.5: Adaptation results of the weights w with JÌ (æ, w) and additive excitation input. 


In this case, convergence tow = w* and T = 1 is achieved after 45s. This demonstrates that 
convergence and thus optimality with respect to the original problem (5.3) can obtained even 
with a locally non-strictly convex objective function JE (æ, w) in (5.53), provided that there 
is an additive input signal which ensures a persistent excitation of a(t) and thus a persistent 
movement of the value of JE (x, w) such that w* becomes the unique minimizer for all t. 


5.2.5 Stability of the Closed-Loop System 


The combination of the open-loop input-state-output PHS (5.3b), the extended CLF (5.27), 
the MOC law (5.29), the adaptation procedure (5.53), and the shorthand notation (5.30)-(5.34) 
results in the following closed-loop system: 


#2 (F(a) — Rte) ante) + Glau", (5.55a) 
u av 
uw IT, Tw), A, (5.55b) 
O 57 = 0, 
LER (5.556) 
Ow 
Lo = Lo, (5.55d) 
wo = Or. (5.55e) 


To perform a stability analysis of the equilibrium (0, w*) of (5.55), V (æ, w) is used as a 
Lyapunov function candidate to prove that 


V (x, w) £ (0,w*): V(x, w) > 0, (5.56) 
V(x, ,w) ER” xR”: V(x, w) <0. (5.57) 
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While the proof of (5.57) can be handled by the following Lemma 5.21, statement (5.56) 
(detailed later in Lemma 5.23) requires some additional preparatory work. 


Lemma 5.21. Consider the closed-loop system (5.55) starting at (£o, wo) € R” x R”. 
Then 


Y (a,w) € R” x R”: / < (5.58) 


ie. V(a, w) decreases monotonically over time. 


Proof. Applying the chain rule to (5.27) and inserting (5.55a), (5.55b), (5.55c), we get 


vlew) = (SE) DE + gan 
AD T 


a ð ð TI u(®, 
A) 6m 
i 
FE) G(a)S-"( er (5.60) 
_ (OV(a,w)\" ƏH (AV(a,w)\' 9I,(x,w) 
ee) Oe 
gt, fet VY + 
T 1 
SY 
=fr - fr - (fr)? +r St 
=- 1 / (f) + Qr: ST- (5.61) 


Since S% > 0 and Q% > 0 holds (see (5.32) and (5.33)), V(x, w) is nonpositive for all 
(x, w) € R” x R”. 


Next, statement (5.56) (positive definiteness of V (x, w)) has to be evaluated. Despite the fact 
that H (æ) is positive definite by definition, V (x, w) = H (æ) + w! E(x) may be nonpositive, 
if “+w! E(x)” is negative for some (x, w) € R” x R”. In particular, we thus have to prove 
that V (x(t), w(t)) is still positive for allt > to with a # O. This is stated in Lemma 5.23 
with the help of the following Lemma 5.22. 


Lemma 5.22. Let V*(x) = H (æ) + (w*)"E(x) be the value function of optimization 
problem (5.3) and let the conditions of Lemma 5.19 hold with wo € M. Then, all weighting 
factors w(t) asymptotically converge to the optimal ones, i.e. w(t) fulfills 


‚im |lw(t) — w*|l, = 0. (5.62) 
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+V Iw (w) A w2 


Figure 5.6: Contour plot of Jw (x, w) for fixed x. 


Proof. If the conditions of Lemma 5.19 hold with wo € M, then Jw(æ, w) is strictly convex 
with respect to w in an open neighborhood M of the optimizer w* for each arbitrary but 


fixed a, i.e. for all t > 0 we have 


OFw (x(t), w) 


(wı(t) — wa(t))" a II (a(t), w) 


hi ðw 


With w(t) = w(t) and wa(t) = w*, (5.63) reads as 


wo -wy (ee )>o 
Ow w(t) 


Insertion of (5.55c) in (5.64) yields (w(t) — w*)" 


equivalent to 
(w(t))' w(t) <0. 


By using the chain rule, (5.65) can be transformed to 


Fo {C(t HLH} <0, 


Se 


) T 
w(t) 


w(t) < 0. With w(t) := w(t) - w 


(5.63) 


(5.64) 


* this is 


(5.65) 


(5.66) 


where (w(t))' w(t) = ||w(t)||$. Multiplying (5.66) by two and applying the square root on 
both sides, we finally obtain 4 ||@(t)||2 < 0, i.e. the distance ||w(t) — w*||2 decreases strictly 


monotonically with time for all t > 0. This results in (5.62). 


With the help of the above Lemma 5.22, we can prove that V (a(t), w(t)) is indeed a positive- 


definite function: 
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Lemma 5.23. Let V*(x) = H(a) + (w*)'S(a) be the value function of optimization 
problem (5.3) and let the conditions of Lemma 5.19 hold with wo € M. Then 


Vt>0: V (a(t), w(t)) > 0. (5.67) 


Proof. Let W+ = {we R” : V(a,w) > 0 Va # 0} denote the set of w € R” where 
V(x, w) > 0 is fulfilled for all x 4 0. As it trivially holds that w* € W* and 0 € W*, and 
as W* is an open set (see Lemma D.2 in Appendix D.1), it can be concluded that 0 € int(W*) 
and w* € int(W*), which is illustrated in Fig. 5.6°°. Hence, there exists an £ > 0 such that 
the ball B(w*,e) = {w € R” : ||w — w*||2 < £} lies completely within W*. According to 
(5.66), ||w(t) — w*||, is strictly decreasing with time. Consequently, there exists a T > 0 
such that ||w(T) — w*||2 = £, i.e. x intersects the surface of the ball. Since ||w(t) — w*||, is 
strictly decreasing, w(t) will remain within the ball for all t > T, thus 


Vt>T: w(T) e Wr. (5.68) 


Since W* is the set of parameters w where V (x, w) is positive definite for all x € R”, (5.68) 
implies that V(a(t), w(t)) > 0 holds for all t > T. With V (æ(0), w(0)) = V(a,0,) = 
H (ao) > 0 and due to the fact that V(a(t), w(t)) is continuous and V (a(t), w(t)) is 
monotonically decreasing according to Lemma 5.21, it holds that V (x(t), w(t)) > 0 for 
allt > 0. 


As a consequence of Lemma 5.21 and Lemma 5.23, V (a(t), w(t)) is a suitable Lyapunov func- 
tion. With this finding, the main statement of this section regarding stability and asymptotic 
stability of the closed-loop equilibrium can be formulated. 


Theorem 5.24. Let V*(x) = H(x) + (w*)'S(a) be the value function of optimization 
problem (5.3) and let the conditions of Lemma 5.19 hold with wo € M. Then x = 0, w = 
w* is a stable equilibrium of (5.55). 
If additionally one of the following conditions holds: 

° 


(1) The autonomous system & = (J(x) — R(x))2H is asymptotically stable with 
y: ymp y 


Ox 
respect to the origin x = Q, 


(2) G(a) has full rank, 


then x = 0, w = w* is an asymptotically stable equilibrium of (5.55). 


68 Note that 0 € int(W*) and w* € int(W*) does not necessarily imply that w(t) € WF holds for all t. 
Fig. 5.6 shows an illustrative contour plot of Jw(æ, w) for a fixed x. Of course, w* = arg min Iw (x, w). 
However, depending on the shape of Jw, it may be possible that the descent direction — V Jw is pointing out of 
WF, which may yield w(t’) ¢ Wt for some t’ > t. Despite the fact that w(t) may be temporarily outside of 
W+, it follows from (5.68) that for a sufficiently large but finite T > 0, w(t) always lies within WT. 


5.2 Single-Player Case 131 


Proof. According to Lemma 5.23, V (a, w) is positive definite and according to Lemma 5.21, 
V(x, w) is negative semidefinite. As such, V (a, w) is a Lyapunov function for the equilibrium 
(0, w*) of (5.55), which is consequently a stable equilibrium [Kha02, Theorem 4.1]. 


To prove asymptotic stability of (0, w*), recall Lemma 5.22, which states that w(t) converges 
strictly monotonically to w*. Now let E = {x € R” : V(x(t), w*) = 0} be the set of states 
where V (a, w) is constant and w = w*. With regard to the individual summands in (5.61), 
we get E = {ze R” : (fr =O)A((Qr = 0)V (5, =0)), w = w*}. With Qy = L(x) > 0, 
statement Q% = 0 is equivalent to x = 0, which implies fẹ} = 0. Accordingly, E can be 
simplified to E = {ze R” : (a = 0) v ((fr = 0) A (St = 0)), w = w*}. Based on this 
definition, conditions (1) and (2) of Theorem 5.24 are then obtained as follows: 


(1) According to LaSalle’s invariance principle, all trajectories a(t) with V (a(t), w(t)) = 0 
converge to the largest invariant set contained in E [Kha02, Theorem 4.4]. With 
(5.55b), S4. = 0 implies u = 0. Due to the assumption that the autonomous system 
is asymptotically stable with respect to x = 0, the largest invariant set in FE is a 
singleton. Thus x = 0, w = w* is an asymptotically stable equilibrium of (5.55) [Kha02, 
Corollary 4.1]. 


(2 


wa 


if G(x) has full rank, then G(x) S~'(a)G' (a) = 0 and hence S% = 0 only holds for 
x = Q. Thus E is equal to {0}, which implies that x = 0, w = w* is an asymptotically 
stable equilibrium of (5.55) [Kha02, Corollary 4.1]. 


Example 7 (Synchronous Motor) 


We apply the presented method to the nonlinear model of a non-salient permanent magnet 
synchronous motor (PMSM) in rotating reference (dq) frame from [OPA* 11]: 


Lsla = —RsIa — QLsIq + Ua, (5.69a) 
Lely = —Relg + AL gla - OQ + Ua, (5.69b) 
JQ. = nppOlg — fQ — Me, (5.69c) 


where Iq and I, are the stator currents, Ua and U, are the stator voltages, Q is the angular 
velocity, Me is the load torque, L is the stator inductance, Rs is the stator winding resistance, 
© is the magnetic flux, J is the moment of inertia, f is the mechanic friction constant, and 
Npp is the number of pole pairs. 


The objective of the control system to be designed is to follow a desired angular velocity Q4°S 
with zero flux current I$" = 0 for a given load torque MẸ. Denote by (UJ®, yg, 10) 
the resulting open-loop equilibrium values and define æ := F(Iga-id®, I, gaes; n-ode)T 
with F := diag{Ls, Ls, J/npp}, and u := (Ua, Uq, Me)". Then, the error dynamics of 
(5.69) can be modeled by the input-state-output PHS (5.3b), where G = diag{1,1,—1/npp}, 
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H(z) =1/2(a'F'zx), 


L Lgl 
0 m ae L,ndes = 
J(x) = — een — 1,046 0 -8 |, 
. L,I% 
-5 (6) 0 
Le Its 
R 0 a 
R= 0 R; 0 
des 
une y ai 
2 Npp 


We choose Ls = 0.5, Rs = 0.5, O = 1, J = 2, f = 0.1, npp = 2, and consider the optimal 
control problem (5.3), where 


ug tudes 


pn Wy w2 w3 wa ws we ri 

0 
3-02 a 
-0.4 a = 
987 | | | | | | | J 
0 S 10 15 20 25 30 35 40 


o | | | | | | | J 
0 S 10 15 20 25 30 35 40 
tins 


Figure 5.7: Controller performance for PMSM example. 
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Since the condition of Corollary 5.8 is fulfilled, the Hamiltonian H(a) is a CLF. The basis 
functions for the extended CLF are chosen to E(x) = (x?, 23, 22, 2172, £283, z1£3)!. For the 
above optimization problem, it can be shown that the exact solution of the HJB equation (5.24) 
is given by V*(x) = 0.5(a7 +23 +223), which implies w* = (—0.5, —0.5, —0.25, 0,0, 0)". 
Now the control system (5.55) is initialized at x9 = 1, wo = O. The shifts in Jw are set 
foc, = 0, c2 = (10,0), = (0,1,0)", c4 = (0,0,1)", c5 = (—1,-1,0)", ce = 
(0, -1,-1)', cz = (-1,0,—1)T, and the learning rate is set to a = 0.5. Att = 10 and 
t = 30, the load torque M{°® increases by 1, and at t = 20, the desired angular voltage 04° 
increases by 0.5. 

Fig. 5.7 shows the trajectories of x(t), u(t), w(t), and Y. It can be seen that after 2s, 
the learning process is completed and both w and Y have reached their optimal values. 
Correspondingly, the trajectories of x and u converge to the solution associated to the optimal 
controller, once the learning process is completed. This demonstrates that the proposed control 
strategy is capable of adapting the optimal parameters after a single learning phase. 

To investigate the effects of an incorrect choice of basis functions, we repeat the simulation with 
= (x) = (x}, 23, 03,2122, 2273, 21%3)', Le. H(x) + w! E (x) does not fit the structure 
of V*(a) and hence Assumption 5.15 is violated. The results are shown in Fig. 5.8. 


z3 tas 


Figure 5.8: Controller performance if Assumption 5.15 is violated. 
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The trajectory of Y reveals a remarkable and distinct oscillatory behavior. However, after 5s, 
the weighting factors w converge to a certain value w°. After the step increases of MẸ and 
Ne, the weighting factors do not remain at their previous values. Both the oscillation of Y (t) 
and the fluctuation of w(t) after the disturbances imply that H (æ) + (w°)! E' (æ) is not 
equal to the value function for this specific set of basis functions. However, the results show 
that even if &' (a) is not accurate, the proposed controller is able to learn suitable weighting 
factors for a “suboptimal” control, with the oscillation of Y (t) being interpreted as an indicator 
of lack of fitness for the chosen basis functions in &' (æ) (see the discussion in Remark 5.13). 


5.2.6 Summary 


In this section, we have introduced a continuous-time adaptive feedback control scheme 
for dynamic optimization problems with generalized Lagrangian performance indices (A.8) 
and general input-state-output PHSs (A.13). Employing the initial value function guess 
V (a) = H(a) generates an admissible controller which is already stabilizing. While applying 
the CLF-based MOC approach to PHSs, this section shows that the port-Hamiltonian modeling 
gives access to a natural CLF candidate, which can serve as a helpful tool for constructing 
universal formulae for stabilization [Son89, CB04]°’. 


Subsequently, a gradient adaptation of the critic weights w based on the Lagrange multiplier 
Y(t) enables a learning of the (unknown) value function V*(a) and thus the optimal control 
strategy u*. Asymptotic stability of the closed-loop equilibrium is shown by the main Theorem 
5.24. Although a reasonable choice of basis functions is nontrivial, simulations show that 
even if (x) is inaccurate, the proposed controller is able to stabilize the system and provide 
near-optimal solution trajectories. Furthermore, the optimality of the adapted control law 
can be assessed for the case of a bad choice of basis functions via Y(t). However, a rigorous 
perturbation analysis for systems where parameterization =(ax) does not fit the structure of 
V*(a) remains an open research question. 


5.3 Multi-Player Case 


If a dynamical system is controlled by two or more actuators with differing interests and goals, 
a scenario emerges which can be modeled as a noncooperative differential game (see Definition 
A.2 in Appendix A.2). This section presents an adaptive controller for such noncooperative 
differential games with dynamics modeled as general nonlinear input-state-output PHSs. The 
proposed controller is able to learn feedback Nash strategies (cf. Definition A.6, see also 
Lemma A.8) for general Lagrange-type performance indices (A.8). 


Subsection 5.3.1 formalizes the problem class by extending the single-player case from Section 
5.2 to N players, and briefly discusses related work. Subsection 5.3.2 provides an analytical 
solution of the so-called modified differential game. Subsection 5.3.3 analyzes the asymptotic 


69 Sontag’s universal formula for stabilization itself is a special case of the modified optimal controller (5.8). It can 


= 
be obtained by choosing (æ) = (3%) G(x)(S(a))—!GT (a) a and S(x) = I. 
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stability of the closed-loop equilibrium resulting from the modified differential game. Sub- 
section 5.3.4 presents necessary and sufficient conditions for H(a) to serve as admissible 
CLF. Subsection 5.3.5 develops an adaptation strategy which enables to learn the original 
(non-modified) differential game (A.8). Subsection 5.3.6 proves the asymptotic stability of the 
resulting feedback Nash equilibrium and gives an illustrative example. Finally, Subsection 
5.3.7 summarizes the main result of this section. 


5.3.1 Problem Definition and Related Work 


Definition 5.25 (Multi-Player Optimal Control Problem for PHSs). For alli € P, 
find the feedback control laws u;(x) as the solutions of 


ae 2 f KE + u) eai (5.70a) 


Ui (t) 


N 
subject to ä (J (a(t) - Re) FEO + S>Gi(w(t))ui(t) (5.706) 


i=l 


with set of players P = {1,...,N}, state vector x € R”, input vectors u; € R”, 
skew-symmetric interconnection matrix J (x) € R"*", positive semidefinite dissipation 
matrix R(x) € R”*", input matrices G(x) € R”*Pi, positive definite and radially 
unbounded Hamiltonian H : R” — R>o; smooth, positive definite 4; : R” — R>o and 
positive definite S;(a) € RP'*Pi. Without loss of generality, we assume that H(a) has its 
minimum at x = 0 with H(0) = 0. 


To find a feedback Nash strategy for (5.70), however, a system of HJB equations [Bre11] has 
to be solved in order to get a set of coupled value functions V} (æ) (one for each specific 
player), cf. Lemma A.8. Since an analytical solution of this problem is currently not known, it 
is crucial to exploit the structure of the optimization problems and/or the underlying plant 
system. For example, in linear-affine systems with quadratic objective functions (LQ games), 
the value functions are quadratic forms and thus the set of HJB equations translates into 
a tractable set of (algebraic) matrix-Riccati ODEs [PMC79, WSE99, AL13, WHLM18], for 
which extensive research has been undertaken in recent decades, see [Eng06] and [ES13] for 
a profound introduction to solution methods for LQ games. 


For the more general case of input-affine nonlinear systems, methods derived from ADP pro- 
vide viable means for solving the system of coupled HJB equations. In this case, approximators 
(critics) for the players’ value functions are applied, whereby the weights of the critics are con- 
tinuously adjusted based on measurements of the current state vector and the current inputs 
of the other players. Depending on the selected ADP approach, however, initial stabilizing 
weights [VL11, LLW14, MLS14, VMH16] or a suitable Lyapunov function [ZCL13, MNSS18] are 
required to guarantee convergence of the weights and stability of the closed-loop system. This 
makes the deduction of a general methodology difficult, especially for large-scale systems. 
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In the following, we present an adaptive feedback Nash strategy for the class of nonlinear 
input-state-output PHSs. In the previous Section 5.2, it was shown for the single-player case 
that H(a) inherently qualifies as a value function candidate. Again inspired by [SK00], in 
the remainder of this section particular attention is paid to determine the deviation of the 
current suboptimal solution of player 7 from the optimal (Nash) strategy using a certain 
Lagrange multiplier Y;. Based on an evaluation of Y;, the respective critic weights are 
updated simultaneously using a continuous-time gradient flow. Finally, the main theorem 
of this section (Theorem 5.53) will show convergence of the critic weights towards the Nash 
solution and asymptotic stability of the closed-loop system. 


5.3.2 Modified Differential Game 


As with the single-player case, an analytical solution of the original differential game (5.70) is 
generally intractable [Bre11, p. 390ff.], [LVS12, p. 278]. However, inspired by the basic idea 
of [SK00], projection of (5.70b) using a suitable CLF allows to formulate a set of modified 
optimization problems that makes obtaining an analytical solution considerably easier. For 
this purpose, we first introduce the new definition of a CLF in the context of differential games 
and then present an analytical solution of the set of modified optimization problems. 


Definition 5.26 (CLFs for Multi-Player Scenarios). A radially unbounded, smooth, 
positive definite function V; : R” — R is said to be a control-Lyapunov function associ- 
ated to input 7 € P for the system 


zeR”, weRP, ieP, (5.71) 


Remark 5.27. For input-affine nonlinear systems 
N 
t= f(a)+) > Gi(x)ui, zEeR”, w €R”, (5.73) 
i=1 


condition (5.72) is equivalent to 


-\ T A T N 
(SE) ew- = (%) [10+ eu = oe 
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\T 
my 


Lemma 5.28. For alli € P, let V;(x) be a CLF associated to input u, of (5.70). Then an 
exact solution of the modified differential game 


Now the original problem (5.70) is modified by left-multiplying (5.70b) with ( 


(5.75a) 


oV; 


a (5.75b) 


subject to V;(a) = ( 


with £;(a) = 0, S;(a) > 0 is given by 


OV; (2) 


-(S:(@)) GT) 


a 


Ji 


Proof. For each i € P, the Hamiltonian of optimization problem (5.75) is given by 


I (w,u,V) = stile) + su Siu Fe (5.82) 
a er a 61 aH 
= 5 lila) + zu Si(w)ui + T; (2y (u- R)? D jw), 
(5.83) 


where Y; is the (scalar) Lagrange multiplier corresponding to the constraint (5.75b) and 
Y := col;{Y;}. The optimal control už can be computed by 


9%, (&, u, T) 


om =0, (5.84) 
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which yields 
urle) = ~(Si(2)) He) Ti. 6.85 
From the HJB equation for time-invariant systems 
Hi(x,u*, ¥*) =0, ieP, (5.86) 
we get 
o [ƏVN' oH 
VieP: 0= cert (Z) (J R) 
* 21 oV; T Wall OV; 
m) HER) 
[VNT Š a OV; ana 
(2) Vaews@ra@a Zr; 687 
j=l 
j#t 


With the definitions (5.78)-(5.81), this is equivalent to (5.76). 


Remark 5.29. The resulting closed-loop system 


OH OV; ov; 
è= (J - Rz, Aa) ET En SH) Ty, 
Ox Ox Ox 
(5.88) 
0=q(e) + Tigi (æ) + Yidi (æ) + Yı $ aij(@) V5, (5.89) 
0 = q5 (x) + Tog (æ) + T2? (æ) + Ta X` B;@)T;, (5.90) 
#2 
N 
0 = q£) + Ynan(®) + Tran (@) + Tw I, ay; (wT; (5.91) 
j=1 
JAN 
constitutes a differential-algebraic system of equations of index 1. © 


According to Definition 5.26, the CLF property of V;(a) is highly dependent on the other inputs 
u~i. Thus, (5.72) must apply to all possible inputs uı,...,u;,...,un. This is generally a 
very strict requirement, if those inputs are assumed to be unconstrained in any way. However, 
since in the context of the modified differential game (5.75) all inputs are restricted according 
to (5.85), a much less restrictive requirement on the vector (Vi,..., Vyn)! of CLFs can be 
imposed: 


5.3 Multi-Player Case 139 


Definition 5.30 (Admissible Vector of CLFs). A vector V(x) = (Vi(a),...,Vp(a))" 
is said to be an admissible vector of CLFs for (5.73), if (5.72) is fulfilled for alli € P with 
u; (x) = ux(ax) according to (5.85), ie. if 


Va # On: inf { (5 uin} <o (5.92) 


holds for alli € P. 


Remark 5.31. For input-affine nonlinear systems, condition (5.92) is equivalent to 


av;\" 
(SE) eve) =, 


foralli € P, æ € R”. ® 


Since the individual inputs u; (x) are interdependent due to coupling by the scalars Yı,..., 


Ty via the control laws (5.76)-(5.77), it is sufficient to require that V (x) is an admissible 
vector of CLFs instead of the (stricter) requirement that each individual V;(a) is a CLF in 
accordance with Definition 5.26. Lemma 5.28 can thus be relaxed as follows: 


Proof. Trivially, if u; (x) is given by (5.76)-(5.77), then (5.74) and (5.93) are equivalent. 


Suboptimality of the Modified Differential Game 


With shorthand notations u = col;{u;} and Y = col;{Y;}, the solution (u*, Y*) generated 
by the MOC law (5.76)-(5.81) can be characterized as the Nash equilibrium of a differential 
game with scaled objective functions Yr ;(x): 


Lemma 5.33. Let V(x) be an admissible vector of CLFs for (5.70). Then each solution 
(u*, Y) fulfilling (5.76)-(5.77) constitutes a Nash equilibrium with regard to the scaled 
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objective functions 


Ir ie) = I “ay (CEO) + DT Su) dt, ieP (5.94) 


subject to the dynamic constraint (5.70b). 


Proof. For each i € P, the Hamiltonian of the scaled optimization problem (5.94) subject to 
(5.70b) equals 


(5.95) 


where €; € R” is the Lagrange multiplier corresponding to the individual optimization 
problem of player i € P. For the optimal control u$“*!°d*, it has to hold that 


OF scaled (q u &,) 
2 = = Dp; 5.96 
Ou; scaled Or; 
which yields 
us" (2) = —(Si(x)) "CIE. (5.97) 
From the HJB equation for time-invariant systems 
Zar ur, £z) = 0, (5.98) 
we get 
. ; = 1 : *\T OH 1 (ex\T A. : -1nT * 
N 
-(E) I HÄTTE ET; (5.99) 
j=1 
JAI 


Now recall the HJB equation for the modified differential game in (5.87). Define 


Ov; 
Ox’ 


which results in (5.87) and (5.99) being equivalent. Accordingly, each solution Y™ of (5.87) 
corresponding to the modified differential game problem in (5.75) is a solution of (5.98), with 
(5.100) associated to the new, scaled problem (5.94) in Lemma 5.33, and vice versa. By inserting 
£* and Y* in (5.97), we can conclude that u% (æ) in (5.76) is a Nash strategy of player i € P 
for the scaled differential game (5.94) subject to (5.70b), i.e. už (æ) = uSc@led* (æ) holds for all 
ieP. 


& = 


ieP, (5.100) 
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Corollary 5.34. Let V(x) be an admissible vector of CLFs for (5.70). Then each solution 
(u*, Y*) fulfilling (5.76)-(5.77) constitutes a Nash equilibrium with regard to the original 


problem (5.70), if Y} (x) = 1. 


Proof. Trivially, by comparison between the original differential game (5.70) and the scaled 
differential game (5.94) subject to (5.70b), which is enforced by the controller (5.76)-(5.77), it 
follows that all components in Y must be equal to one so that both differential games are 
equivalent. 


Remark 5.35. From Corollary 5.34, it follows that, unless Y (x) converges to a constant value 1 
and remains constant even after a disturbance, the chosen set of CLFs V;(a) cannot be equivalent 
to the players’ value functions. In this case, the resulting controllers (5.76)-(5.77) do not constitute 
a Nash strategy with regard to the original problem (5.70). Conversely, the fluctuation of Y (t) 
over time can therefore again be interpreted as an indicator of lack of fitness of the control strategy 
ux (a) with respect to (5.70) as already reasoned in Remark 5.13 for the single-player case. © 


Example 8 (Nonlinear Differential Game) 


Consider the two-player differential game (5.70) with 


1 f* ( +[16+823 —222 5 
i= a (2 | ip 5 |® +4- ui )dt, (5.101a) 
1 fe ( T ° + 222 a 3 
2 = > x xz+us | dt (5.101b) 
F 2 Jo 25 12 2 
subject to the nonlinear input-state-output PHS 
0 a i +2 ] OH al H 
t= 2| — 2 2| I + ui + ua, (5.102) 
$ 0 3 2| ) dx 0 1 
Sas = <> — 
J (x) R(x) Gi (a) G(x) 


where H(x) = 327 + $23. 


The corresponding value functions of (5.101a) and (5.101b) are 


Vi (a) = 22] — 22122 + 23, (5.103a) 
1 
Ale z” +23, (5.103b) 


which can be calculated by using the converse HJB approach [NP96]. 
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The system (5.102) is initialized at x = (0,0). Fig. 5.9 shows the resulting trajectories of 
Yı(t) and Ya(t) for different vectors of CLFs V (x). 
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Figure 5.9: Trajectory of Y for different vectors of CLFs V (æ). 


In Figs. 5.9a-c, Y (t) is varying with time, while in Fig. 5.9d it holds for allt that Y (t) = 1. 
Moreover, it can be seen that the closer V (x) is to V* (æ), the closer Y is to 1. 

However, V;(a) = V,*(a) does not necessarily result in Y;(t) = 1, which can be seen e.g. in 
Fig. 5.9b: Although V2(a) = Vž (x) holds, Ya approaches a value ~ 1.1. 

This effect is based on the dynamic coupling of both players and the nature of a Nash equilib- 
rium: Nash strategies optimize the individual objective function of each player while assuming 
that the other players also act in an optimal way. However, since V\(a) # Vč (æ) in Fig. 
5.9, the strategy of player 2 is not a Nash strategy which is indicated by Ya(t) # 1 accord- 
ing to Corollary 5.34. The same holds for player 1 in Fig. 5.9c where Vi (x) = Vt (x) and 
Yı ~ 1.017. 


5.3.3 Stability of the Modified Differential Game 


In this section, we show the relation between the sign of Y; and asymptotic stability of the 
closed-loop system (5.70b), (5.76), (5.77). In particular, we show in the next lemma that an 
asymptotically stabilizing controller can always be constructed by choosing a positive solution 
for Y;. Subsequently, we study the existence of a positive solution in Lemma 5.37. 


Lemma 5.36. Let V (x) be an admissible vector of CLFs for (5.70) and £;(a) > 0. Then the 
equilibrium x = 0 of the closed-loop system (5.70b), (5.76)-(5.77) is globally asymptotically 


stable, if Y; > 0. 
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Proof. Let Y; > 0 holds. Inserting u,(z) = už (æ) in (5.82) and application of (5.86) yields 
VieP: Vi (x) = - — 
< —— 4; (x) <0. (5.104) 


Thus for each i € P, the Lyapunov function candidate V; (æ) fulfills V; (æ) + 0 and V; (æ) < 0 
and is radially unbounded, which means that x = O is a globally asymptotically stable 
equilibrium of (5.70b), (5.76)-(5.77) [Kha02, Theorem 4.2]. 


Lemma 5.37. If (5.77) has a real solution, then there exists exactly one solution Y with 
Y, >0. 


Proof. Let Y € RN be a solution of (5.77). For each i € P, (5.77) takes the form 


(a) T? + | + IT; | Ti + ade) =0. (5.105) 
Sa’ jal Sa’ 
k IF = 


—b; xy b? — da;c; 0 
’ a > 

Ñ= Day (5.106) 
ae Qi = 0. 


Due to the fact that q? (x) > 0, the inequality c; > 0 holds for all x 4 ©. Moreover, a; x 0, 
which follows directly from (5.80). Thus if a; < 0, then the “—” solution in the first row of 
(5.106) always leads to Y; > 0. Otherwise, if a; = 0, then with the CLF property (5.93) of 
V;(a) we get 


T N 
i= (=) f(x) +) G;(æ)u; | <0, (5.107) 


which again results in Y; > 0. 


In the following, we assume that the positive solution Y+ of (5.77) is chosen. We finish this 
subsection with two concluding remarks. 


Remark 5.38. The continuity of Y ;(a) can be shown by applying de L’Hospital’s rule to (5.106). 
© 
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Remark 5.39. By using the alternative Lyapunov function candidate )_,.p Vi(x), it can be 
shown that asymptotic stability is still preserved, if l;(x) > 0, £-;(&) = 0, and correspondingly, 
if Y} > 0, T}; > 0 holds for some i € P. However, for the sake of brevity and to avoid 
case distinctions in the proofs, we will proceed with the stricter requirement €;(a) > 0 for all 


ieP. © 


5.3.4 Admissible Vectors of Control-Lyapunov Functions for 
Input-State-Output Port-Hamiltonian Systems 


As mentioned above, finding an appropriate CLF for a given system is in general very difficult, 
even in the single-player case. Therefore, in this subsection, we will investigate under which 
condition the Hamiltonian H(a) can be used as a CLF V;(æ), in particular, under which 
conditions the vector H(a) - 1 is an admissible vector of CLFs. 


Lemma 5.40. The vector V(x) = col,{H(ax)} = H(a) - 1 is an admissible vector of 
CLFs for the input-state-output PHS (5.70b), if and only if 


ad aH 


n 
>> ( ) AICHACH 5 Y 0; 


TAI 
(5.108) 


Proof. With AG, as in (5.109), condition (5.93) is equivalent to 


av,\" = <0, 2#0n, 
VieP, re AX@,: (Z) f(a) +X G;(a)u; eae, (5.110) 
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the vector Vo(a) = col;{H(x)} = H(x) - 1 is an admissible vector of CLFs for the input- 
state-output PHS (5.70b), if and only if (5.108) is fulfilled. 


Corollary 5.41. The vector Vo(x) = col;{H(a)} = H(x) - 1 is an admissible vector of 
CLFs for the input-state-output PHS (5.70b), if and only if (5.70b) with passive outputs 


OH (x) 
= Ox 


(5.112) 


is zero-state detectable. 


Proof. Let Vo(z) = col;{H(a)} = H(x) - 1 be an admissible vector of CLFs for the input- 
state-output PHS (5.70b). Then (5.108) is fulfilled (and vice versa). Due to the fact that 
XG, ={xeER”: G] (a) 4 = 0 x Æ 0}, (5.108) is equivalent to 

Va € {x € R” : y,(2) =0}: H(a) <0. (5.113) 


With the Lyapunov function candidate H (æ) > 0, this means that (5.70b), (5.112) is zero-state 
detectable (cf. [van17, p. 47]). 


Although the lemmas presented so far are necessary and sufficient, they are generally difficult 
to verify. Therefore, the following lemma presents a more practical condition which can be 
directly evaluated based on the given system and input matrices in (5.70b): 


Lemma 5.42. The vector Vo(&) = col;{H(x)} = H (x) -1 is an admissible vector of 
CLFs for the input-state-output PHS (5.70b), if H (x) is pseudoconvex and at least one of 
the following conditions holds: 


1. The dissipation matrix R(æ) has full rank n. 


2. At least two input matrices Gi, (£), Gi, (x£) have full rank n. 


Proof. Since H(a) is pseudoconvex, there exists no x’ # 0 such that VH (x') = 0. 
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1. Frank{R(z)} = n, then R(x) > 0 and thus 


(=) Fla) +Y ce; 


- (2) uo Ray) DE) ee, 


OH\" BR, Sven ir, „OH 
=- (3) Roge (22) HOCH NH Fee 
>0 as >0 


is always fulfilled for all ¿ € P, which implies that (5.108) is fulfilled. 


2. If at least two input matrices G;, (x), Gi, (ax) have full rank n, then for all i € P there 


exists at least one k Æ i such that G,.(x)($,(x))"'G} (x) > 0. Accordingly, for all 
i € P the inequality 


(E) uw- Rey) (ers, 


N T 
-> (£) ET, <0 (5.115) 


holds, which implies that (5.108) is fulfilled. 


In the following, Lemma 5.43 gives a necessary and sufficient condition for the special case of 
(5.70b) being a linear input-state-output PHS: 


Lemma 5.43. For the linear input-state-output PHS 


N 
&=(J-R)Qxt+ > Giu; (5.116) 


i=l 
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with H(x) = 32!Qxz,Q > 0, the vector Vo(x) = coli {H(x)} = H(a) - 1 is an 
admissible vector of CLFs, if and only if 


N N N 

ker(R) N (N ch) Nn | U 0) ker(@;57'G|) | =9. (5.117) 
i=l i=1j=1 
jzi 


Proof. (see Appendix D.2). 


5.3.5 Adaptive Differential Game 


From the previous subsection, it can be concluded that the application of the modified differ- 
ential game (5.75) with V (a) being an admissible vector of CLFs always allows the derivation 
of stabilizing controllers u% (æ) which constitute a Nash strategy with respect to (5.94) and 
(5.70b). However, since Y; depends on V (a), the condition Y; = Y; = 1 in Corollary 5.34 is 
in general not fulfilled for all ¿ € P. Thus, according to Corollary 5.34, the modified differential 
game is not equal to the formulation (5.70). 


To eliminate this discrepancy in the same spirit as for the single-player case, we thus introduce 
a vector of extended CLFs V (x, w) with additional weighted sum of basis functions. The aim 
is to achieve Y;(t) = 1 for all i € P, so that according to Lemma 5.33, the scaled objective 
functions (5.94) being minimized by the resulting modified optimal controllers (5.76)-(5.77), 
are identical to the original objective functions (5.70a). 


Adaptation using Extended CLFs 


Each extended CLF V;(a, w;) is supposed to be the sum of H(a) and a weighted sum of 
ri € IN basis functions 


V, (a, wi) = H(a) + w]S,;(«), (5.118) 


where ©; : R” > R” and w; € R”. We assume that all basis functions &;(a) are smooth 
and “properly chosen” in the sense that the (optimal) value functions V;*(a), i.e. the solutions 
of the corresponding HJB equations 


H (ew : ) =0, (5.119) 
axe 


Assumption 5.44. For each player i € P, there exists aw* € R™ such that 


)'Ei(@). 
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Analogously to Remark 5.16, Assumption 5.44 is justified by the Weierstraß approximation 
theorem whenever the number of basis functions is large (see also [VL11, p. 1559]). 


Now the aim is to set up w; and thus V;(x, w;) in such a way that w; = wX in order to 
enable that V (x, w) = col; {V;(x, w;)} constitutes a vector of value functions for (5.70), i.e. 
a solution of the set of HJB equations (5.119) associated to the original problem (5.70). This 
property can be checked using the following corollary: 


Corollary 5.45. For alli € P, let V;(x) = H(a) + (w?)'Z,(x) be a given CLF 
with w? € R™. Then V(x) is equivalent to the vector of value functions V* (x) = 
col; {V;*(a)} of (5.70), if and only if 


VieP,xzeR”: (z,w?) € QO; (x, wi), (5.121) 


where 


QO; (x, w) := { (a, wi) E€ R” x R”: wl A;(£)wi + al (x) w; +a;(x) = 0} i 


az; T 
+( 


OH 
a(i 


Proof. (see Appendix D.2). 


Lemma 5.46. Foralli € P, let 


Q(x, w;) := wl A;(x)w; + al (x) w; + a;(a) (5.127) 


with A;(a), a;(x) and a;(a) as in (5.123)-(5.125) be the corresponding quadratic function 
of the quadric Q; (x, w;) in (5.122). Then for each arbitrary but fixed x € IR”, the objective 
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function 
Iui@, wi) =) (la + ci)? (5.128) 
j=l 


consisting of a sum of squared quadratic functions Q; whose arguments are shifted by 
Grein € RN, has a global minimizer at w; = w% and is locally strictly convex in 
an open neighborhood M; of w%, if and only if the matrices 

2(w}) "Aj (a + ci) + a] (2 + ci,1) 
A; := (5.129) 


(w) TA; (a + Cisr) + a] (2 + Cir) 


have full rank ri. 


Proof. The proof is a direct consequence of the proof of Lemma 5.19. Here it has to hold for 
each player separately. 


Remark 5.47. Letw = col;{w;} with w € R” andr = rı +---+1y denote the concatenation 
of all weights wi, i € P and let Ty(x,w) = >I cp Iw, i(£, wi) be the sum of all objective 
functions Jw (x, w). Then 


w* = argmin {Ĵ„ (æ, w)}. (5.130) 


w 


© 


Remark 5.48. For simplicity of notation, it is pointed out that the condition in Lemma 5.46 
concerning local strict convexity of objective functions Jw i(w;) in an open neighborhood M; 
of w% is equivalent to the statement that the compound objective function Tn (a, w) is locally 
strictly convex in an open neighborhood M = M, x --- x My ofw*. © 


With Remarks 5.47 and 5.48, we can again apply the unconstrained gradient flow to adapt w, 
which finally leads to 


ie Sigg OY. een) (5.131) 


where a > 0 specifies the learning rate. 
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Closed-Loop System 


The system dynamics (5.70b) together with the adaptation procedure (5.131) result in an 
extended system 


N 

& = (J(x) — R(x)) ante) + S°Gi(x)ui, (5.132a) 
i=l 

w; = —a- Iw i (2, Wi) ieP. (5.132b) 


Ow; 


With (5.118), we obtain 


dV;(a,w;) OH (=) 


ae = wi (5.133) 


and accordingly the modified differential game for the extended system (5.132) reads 


tr 
VvieP: min {5 L;(2) + (u) Sj (a) a, a) (5.134a) 
U; 0 
_ (Næ wN S (Vi(w,w)\" 
s.t. Vi(e.w) = (E 2+), Fu, ) Wir (5134) 


For this modified differential game, it is straightforward to obtain the explicit control law 
from (5.9) 


uf = -(S:(2)) IT) Til wi), (5.135) 
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where 
Yi £ Vea? + Or: , Si 
1 ’ Sy, #0, 
T(x, w;) = , STi (5.136) 
= Ti 4 — 
2fri a 
avi\" ðH (wy fav" av; 
! = l £ j Zul, An 
fra(ew) = (SE) Ue- Re) 2 me) Kar, 
jżi 
e T 
Ow; Ow; 
T: 
= (5) tar) 
z£ Ox 
oH WY ðH (08;\" 
j EEN l 
x (uow Re) E Ko (2 + (=) w) ) 
jżi 
IVN OTi 
= (=) PS (5.137) 
öH\' 1,05, ðH w= 
-((&) tul? K DR +5 Gila 
jżi 
VIN Twi 
= (=) T (5.138) 
av; \ ƏV; 
Stile w) = (GE) Ke 
ðE =;\" OH\" 0B; N 
N 2 i Be i 
=w Ki (=) w2 (2) Kæ (=) wi 
ƏHN' OH 
+ (5) Kıla) zn (5.139) 
Qr (2) = F(x), (5.140) 
K; (a) = Gi(x)$; *(x)Gi (æ). (5.141) 


Note that the “+” sign in the first row of (5.136) can be discarded with the same reasoning as 
in the proof of Lemma 5.37. Moreover, continuity of Y;(a, w;) is still preserved (cf. Remark 
5.38). Finally, (5.137) and (5.138) are equivalent as it follows from (5.135) that 


OH , (08;\'" 
Y;: K;(«) (z + (=) w) = —G;(x)uj. (5.142) 
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Remark 5.49. From the controller equations (5.135)-(5.136), (5.138)-(5.141) it becomes evident 
that neither E; (x) nor wj of the other players j # i must be known byi € P. © 


By combining the modified optimal control law and the adaptation procedure (5.131), we get 
the closed-loop system 


as OH Č aan aAA 
& = (J(x) - R(x)) 5, = >, Yi Gi(x)(Si(x)) G; (x) Dar? (5.143a) 
wi = —a OTe (5.143b) 
Ow; 
N 
j=1 
Jri 
(5.1430) 
x(0) = £o, (5.143d) 
w;(0) = 0,,, (5.143e) 
where 
oP (@) = ie), (5.144) 
OVN Br 
df 1 = SEAN, g W, 
qi (x, wi) = qi (x) (=) Ow; ’ (5.145) 
Pilea. Pook OMEN) une ea 
G(w,wi)=—5(5*) Gilw)(Silw)) GT OT 6140) 
au" _ av; 
Gi; (@, wi, wg) = (=) G;(x)(S;(a)) Home (5.147) 


5.3.6 Stability of the Adaptive Differential Game 


To perform a stability analysis of the equilibrium (0,,, w*) of (5.143), we define the Lyapunov 
function candidate 


V(x, w) :=) Vi(a, w) (5.148) 


and show that it is indeed a Lyapunov function for the closed-loop system (5.143). 


Lemma 5.50. Consider the closed-loop system (5.143) starting at (xo, 0,). The Lyapunov 


function candidate V (a, w) is strictly monotonically decreasing over time unless (x, w) = 


(On, w*). 
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Proof. The time derivative of V; (x, w;) equals 


; OVANT. AN & 
ew) = (Z) e+ (2) w 


5 (2) (I(x) - R(w)) Z - T4- (E) aoso 


N əv, x F | 
z > Tj: (&) G;(2)(S;(&)) G! (a) ae 
IFi 


Dividing (5.143c) by T; > 0 and inserting V; from (5.149) yields 


14) | 7 | Y; oV; i . ; -l øT oV _ 
a7, (ta (%) G;(z)($;(&)) G; (x) Ep =, (5.150) 
which finally leads to 
>o lala) Yi (IVN ehe nee io 
V; = 3, ar i) Gilx)(Si(£)) G; (x) dr <0. (5.151) 
Å< wN — amaaa 
>0 >0 =0 


As a consequence, V = > V;(&,w;) < 0, unless x = On. 


Remark 5.51. As already mentioned in Remark 5.39, it is sufficient for negative definiteness of 


V(x, w) (see (5.151)) and thus asymptotic stability of the closed-loop equilibrium to require that 
Li(x) > 0, €.i(a@) = 0 holds for some i € P. © 


Lemma 5.52. Consider the closed-loop system (5.143) starting at (ao, Or). Let the con- 
ditions of Lemma 5.46 hold with wo € M. Then the Lyapunov function candidate 
V(z,w) = 1'V(a, w) as in (5.148) fulfills 


Vt>0,(a,w) 4 (On, w*): V (a(t), w(t)) > 0. (5.152) 


Proof. V (a, w) can be written in the form 
V(a,w) = N-H (x) + w'E (a), (5.153) 
where =(a) = col;{=;(a)}. Due to the fact that both V (æ, 0,) > 0 as well as V(x, w*) > 0 


hold for all x € R” and J, (x, w) is strictly convex, the conditions of Lemma 5.22 are fulfilled, 
which implies that 


lim ||w(t) — w* lo = 0. (5.154) 


Accordingly, from Lemma 5.23 it follows that V (a(t), w(t)) > 0 for all t > 0. 


154 5 Optimal Control of Port-Hamiltonian Systems 


Theorem 5.53. Consider the closed-loop system (5.143) starting at (xo, 0,). Let the 
conditions of Lemma 5.46 hold with wo € M. Thenx = 0,w = w* is a locally 


asymptotically stable equilibrium of (5.143). 


Proof. Define the shifted Lyapunov function candidate V (æ, w) = V(x, w — w*). According 


to Lemma 5.52, V (x, w) is positive definite and according to Lemma 5.50, V (a, w) is negative 


definite. As such, V (a, w) is a strict Lyapunov function for the equilibrium (0, w*) of (5.143), 
which is consequently a locally asymptotically stable equilibrium [Kha02, Theorem 4.1]. 


Example 8 (Nonlinear Differential Game) - cont’d 


Now the adaptation strategy (5.143) is applied to the nonlinear differential game (5.101)- 
(5.102). As R(x) in (5.102) is positive definite, Lemma 5.42 states that Vo(x) = 
(H(a), H(a))' is an admissible vector of CLFs. 


The basis functions for the extended CLFs are chosen to By (x) = Es(®) = (x7, 2122, 273)", 
which, after comparison of (5.103) and (5.118), implies that the optimal weights are w{ = 
(1.5, —2,0.5)" and w = (0,0,0.5). 


The system is initialized at £o = (1,1)', wo = (0,0)" and the shifts in Tw; (see (5.128)) are 
set to c;ı = (0,0)', ci2 = (1,0)', c;3 = (0,1)', and ec; a = (—1,—-1)". To investigate 
the behavior of the control system in case of an external excitation, we apply an additive 


disturbance input d = (ö(t — 10), 6(t — 20))". 


Fig. 5.10a shows the trajectories of w(t), wa(t), and Y (t), if the learning rate a is set to 0.5, 
with the dashed lines indicating the optimal weights wï and w3, respectively. It can be seen 
that after 2s, x has converged to its equilibrium value and both Y as well as wı and wa have 
converged to their optimal values. Moreover, their values remain identical once the learning 
process is completed, even after the additive disturbance at 10s and 20s. Thus, it can be 
deduced that Vı (x, wı) and V2(x, w2) converge to the respective value functions V¥ (x) and 
V3 (a), which means that the implemented control strategies ui (x) = men . Gi (a) “71, 
and u(x) = — 5 - G} (a) av, converge to feedback Nash strategies with regard to the 
original objective function (5.70a). 


Fig. 5.10b shows the trajectories, if œ is reduced to 0.05. In this case, the equilibrium values of 
wı, wa, and Y remain the same. However, convergence of Y, wı, and wa is considerably 
slower, with a time to convergence of around 20s. The overshoot of Y is about ten times larger 
than in Fig. 5.10a. The disturbance att = 10s results in a slight displacement of Y, wı, and 
wo, whereas there is virtually no displacement caused by the second disturbance at t = 20s. 
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This again shows that even after a disturbance, w, and wə remain at their optimal values 
once the learning process is completed. 


tins tins 


(a) a = 0.5 (b) a = 0.05 


Figure 5.10: Adaptation results of the weights w and Lagrange multipliers Y depending on the chosen learning 
rate a. 


5.3.7 Summary 


In this section, a novel approach for finding feedback Nash equilibria in N-player nonco- 
operative differential games for input-state-output PHSs has been presented. The proposed 
design procedure is based on a modified differential game which yields Sontag-type explicit 
control laws for all players, provided that an admissible vector of CLFs can be found. First, 
necessary and sufficient conditions have been stated under which the Hamiltonian of the 
input-state-output PHS accomplishes this purpose. Then, in order to ensure the optimal 
controllers for the modified differential game to coincide with the optimal controllers for the 
original problem, a simple adaptation strategy based on gradient flow is applied which is 
proven to be asymptotically stabilizing and able to approximate the value functions of the 
individual players (cf. Theorem 5.53). 
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5.4 Discussion 


In this section, we restate the optimal balancing problem (4.37) from Subsection 4.4.2 and 
provide a conclusive statement about its explicit solution followed by a brief outline of 
prospective avenues of further research. 


5.4.1 Application to Balance of Payments Restoration 


Recall the optimal balancing problem from Subsection 4.4.2, yielding 


1 f“ 
min ; f x K A Zo +u'Sudt (5.155a) 
m 
Q 
OH, (az) 


subjectto 2, =(J,—R,) + Gyu. (5.155b) 


OL 
Note that Q = 0 and S > 0 hold by definition. Provided that the (mild) technical assumption 
of nontrivial zonal power injections p, # 0 holds, the following corollary gives the asymptotic 


stability of the resulting (closed-loop) equilibrium governed by the PHS dynamics (5.155b) 
and the adaptive optimal controller (5.55b), (5.55c). 


Corollary 5.54. Let V} (£o) = Ho(£o) + (w*)'E (a) be the value function of opti- 


mization problem (5.155) and let the conditions of Lemmas 5.19 and 4.19 hold with wo € M. 
Then £o = 0, w = u% is an asymptotically stable equilibrium of (5.155b), (5.55b), (5.55c). 


Proof. If the conditions of Lemma 5.19 hold, then Theorem 5.24 can be applied to state that 
(5.155b), (5.55b), (5.55c) is at least (non-asymptotically) stable around (x, = 0,w = w*). 
Moreover, if the conditions of Lemma 4.19 hold, then this lemma states that the (open-loop) 
PHS dynamics (5.155b) are globally asymptotically stable. Thus, asymptotic stability of the 
closed-loop system (5.155b), (5.55b), (5.55c) around (x, = 0, w = w*) follows by Theorem 
5.24. 


5.4.2 Possible Extensions 


The novel methodology for ADP-based solutions of optimal control problems and differential 
games with PHS dynamics is founded on the “Lyapunov-like” properties of the Hamiltonian 
H (a), which allow to obviate some common challenges in ADP literature, such as the need 
for initial stabilizing weights. However, for didactical reasons, a number of simplifying yet 
not necessarily essential assumptions are made in the context of this chapter. In the following 
we thus briefly discuss worthwhile extensions and generalizations of the previous findings. 
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Extension to Weak CLFs 


A remarkable insight following from Corollary 5.54 is that the CLF property of H (æ) is 
not a necessary condition for asymptotic stability of the closed-loop system. In particular, 
although H,(a,) is only a weak CLF”, the special structure of Ho (£o) ensures that the 
largest invariant set contained in H o(£o) is still equal to the origin, such that asymptotic 
stability can be addressed by invoking LaSalles invariance principle. In general, note that H (x) 
being a weak CLF is also sufficient for non-asymptotic stability of the closed-loop system for 
both the single- and multi-player case’!. However, for asymptotic stability, it always has to be 
investigated “by hand” whether the largest invariant set contained in {a 4 0 : V (æ, w) = 0} 
is equal to the origin. 


Extension to Passive Input-Output Systems 


Although all considerations of this chapter focus on optimization problems in which system 
dynamics can be described by an input-state-output PHS, they can seamlessly be extended to 
the more general system class of passive input-output systems (cf. Definition A.15 in Appendix 
A.4). In particular, all stability statements made so far still hold as long as the storage function 
V (a) of the passive input-output system has the property of a (weak) CLF (cf. Definition 
5.26 and Definition 5.30). A very practical benefit of PHSs, however, is that the Hamiltonian 
obtained there as a direct outcome of the energy-based modeling procedure always reflects 
the actual energy stored in the system. Therefore, H(a) highly qualifies as a natural CLF 
candidate, whereas in (generic) passive input-output systems a suitable storage function has 
to be found artificially by means of educated guessing and thus often has no intrinsic physical 
interpretability. 


Extension of Gradient Flow 


As stated in Remark 3.28, the gradient flows in the adaptation schemes in (5.55c), (5.131) can 
alternatively be replaced by the Newton gradient flows 


w; = a> 0. (5.156) 
If ill-conditioning of the Hessian in (5.156) hinders the numerical calculation of its inverse, 


there is a broad literature on alternative formulations of the Newton descent direction, such 
as regularized Newton’s method [Pol09b] or the pseudo-inverse formulation [GK65] 


27 \Ft ' 
wi = —a- (? Za) Twi a > 0. (5.157) 


70 H(æ)is said to be a weak CLF if the requirement “< 0” in (5.6) is relaxed to “< 0”. 
71 Provided that the conditions of Lemma 5.19 (or Lemma 5.46 for the N-player case) with wo € M continue to 
apply. 
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Besides the standard least-squares pseudoinverse (5.157), there exists a large number of ad- 
vanced approaches based on singular value decomposition such as truncated pseudoinverse 
or damped least-squares pseudoinverse, see [MCA95] for a discussion of alternative formula- 
tions. 


6 Case Studies 


The aim of this chapter is to demonstrate the capability of the overall control scheme shown 
in Fig. 4.7 in its entirety. Section 6.1 introduces the employed benchmark simulation model, 
which is based on the IEEE 57-bus system. Section 6.2 demonstrates the performance of the 
cell-based frequency and voltage controller from Chapter 3 using a three-cell scenario and 
showcases the benefits of the WoC scheme compared to an islanded MG operation (Case 
Study I). Section 6.3 evaluates the efficiency of the overall control scheme by means of a 
ten-cell scenario (Case Study II). The latter Case Study Incorporates the frequency and voltage 
controller from Chapter 3, the price-based controllers for congestion management and BoP 
containment from Chapter 4, and the BoP restoration strategy developed in Chapter 5. Section 
6.4 draws a conclusive statement about the main findings of both case studies. 


6.1 Benchmark Model Setup 


The simulation is carried out on a modified IEEE 57-bus system depicted in Fig. 6.1. As 
part of the IEEE power system test case benchmark systems [IEE21] as well as the MATPOWER 
package [ZMST11], the IEEE 57-bus system was originally derived from a segment of a real- 
world power system localized in the midwestern US. It has already gained notable popularity 
and application in numerous works from different research directions. For instance, [JW16] 
employs the IEEE 57-bus test case to test a probabilistic OPF algorithm considering both 
uncertain infeed and consumption. In [ZC06], the same test system is used to demonstrate the 
effectiveness and performance of a new interline power flow controller approach. In [KPS15], a 
price-based optimization algorithm which minimizes the total cost of active power generation 
by minimizing the sum of active power losses in the network is tested on the IEEE 57-bus 
system. 


Due to its complex structure with 80 power lines, a high mesh degree’? and widely diverse 
line admittances, the IEEE 57-bus system is supposed to be a pertinent basis for the purpose 
of thoroughly validating the controller concepts derived in this dissertation. Yet, we have 
conducted the following modifications and amendments to the original model [IEE21] in order 
to provide a more insightful playground with regard to all possible network connectors and 
participants established in sections 3.2 and 3.3: 


e Originally equipped with only 7 SGs and 42 loads, we have connected 38 controllable 
sources to the network (19 each of S and Z type) as well as one (uncontrollable) load 
(pei; qei) per bus i € V, modeling the connection to local low-voltage feeders. The 


72 The associated graph @prp has an average degree of 2.81. 
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Figure 6.1: Stylized representation of the IEEE 57-bus system [IEE21]. S nodes are depicted in black, Z nodes are 
depicted in gray, and £ nodes are depicted in white. Each bus is equipped with an uncontrollable load 
modelled by (uncontrollable) active and reactive power consumption. 
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Table 6.1: Node parameters. 
Type A; T; Xai — Xai TU,i 
s [12,17 [20,27] [0.12,0.19]  [6.4,7.7] 


T [12,17] [455] = ue 
Lo Vea -— = — 


assignment of buses to node types is illustrated in Fig. 6.1. A detailed list is provided by 
Table E.1 in Appendix E.1. 


Since the node parameters are not specified in [IEE21], they are chosen to be randomly 
distributed within the specific intervals shown in Table 6.1. The parameter values for S 
nodes, £ nodes and transmission lines are based on those provided in [TBD16, KBKH19]. 
The parameter values of Z nodes base upon [MDS* 18]. A full list of all numerical values 
is given in Table E.1 in Appendix E.1. 


Throughout this chapter, all of the following numerical values are given in p.u. with Upase = 


135 kV, Sbase = 100 MVA, and Chase = 1 MU/Sbase- 


All simulations conducted in this chapter are implemented using the software system Wolfram 
Mathematica (Version 12.0.0) with the numerical solver NDSolve. In general, the program 
components developed in the context of this dissertation exhibit a high degree of modularity, 
so that changes to the plant model and variations or partial replacements of the employed 
controllers are straightforward. This enables to conveniently select and evaluate a wide variety 
of combinations of network topologies, parameter ranges, and control strategies. 


The numerical computations are conducted on a machine with an Intel Core i7-6600U and 12 
GB of RAM. 


6.2 Case Study I: Three Cells 


The objective of the first Case Study Is to investigate the performance of the frequency and 
voltage controllers developed in Chapter 3 and to evaluate the economic efficiency of the 
obtained results by examining different cell topologies. 


6.2.1 Overview and Stimulation Signal 


The IEEE 57-bus system is divided into three cells which are each interconnected via multiple 
boundary lines (see Fig. 6.2). Let Digger denote the incidence matrix of the IEEE 57-bus 
system and DÎppg denote the respective incidence matrix if all IÊ l = 8 boundary edges of 
adjacent cells (colored red in Fig. 6.2) are removed. In the following, we investigate the role 
of connectivity of %, and Y as well as the influence of x by examining the following four 
different topological scenarios: 
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Figure 6.2: IEEE 57-bus system divided into three cells. 
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Table 6.2: List of scenarios elaborated in Chapter 6 (CM = congestion management). 


Scenario Dp D. Zonal Pricing BoP 

3 3 
I-a) Dire D IEEE = = 
I-b) Dire Diese = 5 
I-c) Dirre Diese = -— 

3 

I-d) Digger Dire k" = (1, 3, 9)T =S 
I-a) Dirree Dips CM without integral action containment 
II-b) Dire Dior CM with integral action containment + restoration 


I-a) Islanded scenario: All physical connections between different cells via boundary lines as 
well as well as communication links between CCs are cut such that both %, and Y, are 
not (weakly) connected. In particular, we choose Dy = D. = DYspr- This scenario is 
intended to show the behavior of three autonomous MGs, each of which are operated 
in islanded mode and controlled separately by the price-based frequency and voltage 
controller as derived in Chapter 3. 


I-b) Independent price zones: The cells are physically connected, but the communication net- 
works of CCs are separated from each other, i.e. Dp = Diezr and De = Di app. This 
corresponds to the “core scenario” of Section 3.4 where there exists no interconnection 
of price zones. 


I-c) Uniform pricing: Both the physical power lines and the communication networks of 
cells are connected by choosing Dp = D. = Diez, which results in one large cell 
with one centralized CC. 


I-d) Constant participation factors: The cells are physically connected as in Scenario I-b) 
with Dp = Dieer and D. = Den. In addition, CCs coordinate the relationship 
between specific prices based on (3.145) with participation factors K” set to a constant 
value of k” = (1,3,9)'. This portrays the idea of interconnected zonal prices pursued 
in Subsection 3.5. Since no further means of monetary balancing mechanism is active, 
x is set equal to K”. 


A comparison chart summarizing the different scenarios of Case Study I is provided in the 
upper half of Table 6.2. 


Controller Parameters 


The controller parameters in T, from (3.110a) are set to 0.1, while the controller parameters 
in T(.) from (3.110b)-(3.110k) are set to 0.01. The lower and upper bounds of pg; are set to 
p,, = 0.002 and P, ; = 0.003 respectively, and the voltage limits are set to U, = 0.98 and 
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Initialization 


At t = 0, the system is in synchronous mode with L = 0. All controller variables Po V, A, 
}4(.) are initialized to zero. Without loss of generality, we set W = Z and choose the cost 
functions to 


Cr (Pr) z ; í ( 5 £ a) TEW, (6.1) 


where w; = 1 +0.04- (i — 1). The initial values of the voltage angle deviations V;; are chosen 
to be randomly distributed within the interval [—0.04, 0.014] and all voltages U; are chosen 
to be randomly distributed within the interval [0.98, 1.02]. 


Stimulation Signal 


The active and reactive power consumptions pọ and q, are set piecewise constant in order to 
simulate a step change in generation or consumption in specific areas of the network. To this 
end, the initial values of p, and q, are chosen such that the AC power flow equations (3.8) 
are satisfied. Subsequently, step-wise load changes are applied at equal intervals of 5 min at 
the six £ nodes 12, 13, 20, 27, 28, and 43. The selection of these nodes was made to ideally 
cover a wide range of load concentrations and thus to provoke diverse and reverted power 
flows. Nodes 12 and 13 as well as 27 and 28 belong to the same cell and are direct neighbors. 
Bus 20 also belongs to the same cell as 12 and 13, but is no direct neighbor of the latter. Bus 
43 belongs to cell 2 and is situated more remotely. A total simulated time oft = 30 min is 
chosen, which incorporates the following five load jumps: 


e Attı = 5 min, the active power consumption pz 2g in cell 3 increases by 0.015, 


e at t2 = 10 min, the reactive power consumption qe 2g at the same node increases by 


0.015, 


e at t3 = 15 min, the active and reactive power consumption at two additional nodes, 20 
and 27, which are also located in cell 3, increase by 0.0075, 


e at t4 = 20 min, the power consumptions at nodes 20, 27, and 28 are reset to their initial 
values. At the same time, active and reactive power consumption at load nodes 12, 13, 
and 43 in cells 1 and 2 increase by 0.0075 each, 


e at t5 = 25 min, the active and reactive power consumptions of all above-mentioned 
nodes are multiplied by —1 to simulate a reversed load flow. 


A graphical representation of the resulting active and reactive power consumptions is given 
in Fig. 6.3. 
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(a) Active power consumption (b) Reactive power consumption 


Figure 6.3: Step-wise jumps in power consumption (p,,q,) for Case Study I. The thin gray lines indicate the 
(constant) power consumption at all other nodes. 


6.2.2 Numerical Results 


Figs. 6.4 and 6.6 show the resulting nodal prices A, active power generations p,, nodal 
frequencies f and voltage magnitudes U for Scenarios I-a)-I-d), respectively. 


In each of the four scenarios, no nodal price differences within each cell k € {1,2,3} can be 
detected. A close-up plot of the nodal prices for Scenario I-a) after tı and t4 (given in Fig. 6.5) 
shows that the nodal prices converge to a common (cell-specific) price Ay in less than 10s. 


Prices and Active Power Generation 


Scenario I-a) (islanded cells) It can be seen in the upper panel of Fig. 6.4a that due to 
physical decoupling, the loads steps in cell 3 at t1, t2, and t3 only affect the price in cell 3 
itself, while prices in other cells remain unaffected. Likewise, the load jump in cells 1 and 2 
at ts only affects the prices in cells 1 and 2 itself, while the price in cell 3 retains its initial 
value. With regard to the active power generations (see the upper panel of Fig. 6.4b), it can be 
seen that the upper and lower limits p, and p_ are respected except after ts in cell 3 and after 
ts in cell 2, where we see a significant oscillation of all generations around the lower limit. 
Moreover, the active power generations in cell 1 do not converge within the interval between 
t4 and t5. 


Scenario I-b) (free zonal prices) The resulting cell-specific prices shown in the second 
upper panel in Fig. 6.4a predominantly follow a similar course as in Scenario I-a). In particular, 
the load jumps in cell 3 at t1, ta, and tg lead to an increase in the prices in cell 3. Likewise, the 
load jumps in cells 1 and 2 at t4 lead to an increase in the prices in cells 1 and 2. However, 
in contrast to Scenario I-a), load jumps in one cell also affect the prices in the other cells, 
signifying the effect of physical interconnection via inter-cell lines. Moreover, the spread 
between zonal prices is lower than in Scenario I-a). Similar to the price curves, the resulting 
active power generations in different cells are affected by each other and exhibit a lower 
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Figure 6.4: Nodal prices and active power generation for Scenarios I-a)-I-d). 
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Figure 6.5: Synchronization of nodal prices towards a zonal price. 


spread than in Scenario I-a) (see the second upper panel in Fig. 6.4b). This illustrates that the 
other cells contribute to eliminating the imbalance in a specific cell. 


Scenario I-c) (uniform pricing) The second lower panel in Fig. 6.4a reveals that in this 
scenario, all prices quickly synchronize to a common value after each load jump. Accordingly, 
load jumps in one specific cell impact the active power generations, nodal frequencies, and 
voltage magnitues in all cells. This corresponds to the individual active power generations 
being closer to each other, barely reaching their specific upper or lower limits, see the second 
lower panel in Fig. 6.4b. 


Scenario I-d) (constant participation factors) As depicted in the lower panel of Fig. 6.4a, 
the cell-specific prices A% converge to distinct steady-state values, which invariably follow a 
ratio of 1:3:9 regardless of their absolute values. This is in line with the specified participation 
factors k = (1,3,9)'. Accordingly, the individual share of active power generation for each 
cell (see the lower panel in Fig. 6.4b) is distributed proportially to both the weighting factors 
a; in (6.1) and the cell-specific participation factors K, while always respecting the specified 
constraints (Pp, P,) on active power generation as observed in the previous two scenarios. 
For the formation of prices (and thus the resulting active power generations), it is irrelevant 
in which region of the network the specific load jump occurs. 


Frequency and Voltage Control 


The resulting frequency and voltage curves show significant differences between Scenario I-a) 
and the other scenarios (see Fig. 6.6). As evident in the upper panel of Fig. 6.6a, in Scenario I-a), 
each load jump causes temporary deviations of the nodal frequencies in the involved cells, 
which are subsequently regulated back to their nominal value. The maximum overshoot 
during the entire simulated time is +40 mHz. After the steps at t4 and ts, the performance of 
the frequency regulation in cell 1 is poor. By contrast, frequency regulation is maintained 
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Figure 6.6: Nodal frequencies and voltage magnitudes for Scenarios I-a)-I-d). 
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without exception for Scenarios I-b)-I-d), with maximum overshoots of +50 mHz (see the 
other subplots in Fig. 6.6a). 


Fig. 6.6b shows the resulting voltage magnitudes in all cells. In Scenario I-a) (upper panel 
in Fig. 6.6b), all upper and lower limits are met except after t5, where most of the voltage 
magnitudes in cell 2 oscillate around their upper or lower limits. As with nodal frequencies, 
the issues with voltage magnitudes are limited to Scenario I-a), while in the other scenarios 
voltage regulation is consistently provided, i.e. voltage magnitudes are kept within their 
respective limits without any oscillations and/or constraint violations. 


6.2.3 Summary of Case Study I 


The presented case study demonstrates the general applicability of the proposed control 
scheme for the automatic regulation of frequency and voltage by means of price signals. How- 
ever, the detailed results for Scenario I-a) show that WPs in cell 2 are unable to autonomously 
meet the local active and reactive power demand without violating the constraints (3.83b)- 
(3.83c). This result exemplifies that the operation as autonomous MGs creates unnnecessarily 
reduced robustness, unless each MG is over-equipped with instantaneous generation capacity 
at any point in time based on peak demand considerations. As discussed at the beginning of 
Subsection 2.2.1, the latter would be highly cost-ineffective. It can be seen in Fig. 6.4a that 
with islanded MG operation, price jumps only occur in cells where the load changes actually 
occur, and that these price jumps are far more severe than in the other scenarios. On the 
contrary, the WoC-based infrastructure with a physically interconnected overall network 
helps to indirectly couple producers and consumers from different regions and thus to increase 
the robustness of the grid against sudden load changes. Interestingly, deploying the new WoC 
infrastructure comes at almost no additional fixed cost compared to autonomous operation, 
since the (few) additional coupling lines necessary for the interconnection of cells are either 
already in place or can be retrofitted between nearby buses with little effort. 


The cell-specific ratios of the free zonal prices resulting in Scenario I-b) provide a mixture of 
the results in islanded MG operation (Scenario I-a)) and those in uniform pricing (Scenario I-c)), 
but, as discussed in Remark 3.34, do not yet follow an overarching goal. Furthermore, the 
results of Scenario I-d) stress that in general, any given relationships between zonal prices 
can be specified via K. In the subsequent Case Study II, such “non-academic” adjustments of 
x as discussed in Chapter 4, will be examined in detail. 


6.3 Case Study II: Ten Cells 


This case study considers the full control scheme as proposed in this dissertation, with zonal 
pricing through «(t) along with frequency and voltage control. Special focus is put on 
real-time congestion management as well as on the attainment of BoP. 
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Figure 6.7: IEEE 57-bus system divided into ten cells. 
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6.3.1 Overview and Stimulation Signal 


The modified IEEE 57-bus system is now divided into ten cells as illustrated in Fig. 6.7 whereby 
the assignment of buses to the node types and all physical parameters (see Table E.1) are left 
identical to Case Study I. 


The communication networks represented by D. and D, are chosen identically”? to the 
physical interconnection structure, i.e. De = Dp and D. = Dp. We examine the following 
two different real-time pricing scenarios: 


Il-a) BoP containment and congestion management without integral action: The consensus- 
based controller for BoP containment developed in Subsection 4.4.1 is applied with 
G, = G. Moreover, the congestion controller (4.14) from Subsection 4.3.1 is applied to 
enforce congestion management without transient constraint violation. 


II-b) BoP restoration and congestion management with integral action: In addition to the con- 
troller for BoP containment, which is left identical to Scenario II-a), BoP restoration via 
the adaptive optimal controller from Subsection 5.4.1 is activated. Moreover, congestion 
controller (4.14) is replaced by congestion controller (4.15) with integral action. 


A comparison chart summarizing the scenarios of Case Study II is provided in the lower half 
of Table 6.2 in Subsection 6.2.1. 


Controller Parameters 


To ensure sufficient timescale separation (Fig. 4.7), the controller parameters are chosen 
as follows: The frequency and voltage controller (3.110) is given the fastest response times 
by setting all parameters 7(., in (3.110) to 0.01. For the “higher-level” balancing (4.29) and 
pricing distribution controllers (4.11), we choose the components in T, and T¢ to 0.1, and 
the components in T,, to 10. Finally, we set the components in Ty and Tyset to 100 for 
congestion controller (4.14) without integral action, and in T;, and TE to 1000 for congestion 
controller (4.15) with integral action. 


The lower and upper bounds of p, ; are set to Poi = 0 and p, ; = 0.5 respectively, and the 
voltage limits are set to U; = 0.98 and U; = 1.02. The maximum admissible voltage setpoint 
deviation is chosen to U^ = 0.015. For both congestion controllers (4.14) and (4.15), the 
thresholds Ci", m € £p are chosen as C™'" = 0.8. The maximum permissible power flow of 


line 35 — 35 is set to S32°%3_ = 0.2. For the other lines, it is set to ee = 0.1. As state and 
controller weight matrices for BoP restoration, we choose Q; = 4: I, Q2 = 20, and S = 4- TJ, 


73 As shown in further simulative studies in [KBKH19], the concrete choice of De or D. has no influence on the 


resulting steady state, yet only effects on the transient behavior, provided that the respective communication 
graphs Y% or Y are connected. 
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Table 6.3: Step-wise load changes for Case Study II. 


timestep cellk nodei Ap,; Age; 


tı = 15min 1 1 +0.05 +0.01 
tg = 30 min 2 35 +0.03 +0.001 
t3 = 45 min 3 28 +0.01 +0.01 
t4 = 60 min 3 28 —0.01 —0.01 
ts = 75min 2 35 —0.03 —0.001 
tg = 90 min 1 1 —0.05 —0.01 


respectively. The basis functions for BoP restoration are chosen to 


po] 


E(x,) = (efo)? (6.2) 


10 
o (Ik= &) 
and as objective function for the adaptation (5.55c), we use 


N 


N 
Tu (®o,W) = (Az, w))” +) AlE + er, w))? +> (Amer, w))” (63) 


k=1 k=1 


with a learning rate of a = 0.5. 


Initialization and Stimulation Signal 


~ set 
The initial values of all controller states are set to 0, except for u“ (0) = 1. We set W = Z 
and choose the cost functions to 


1 1 
Cx(Dg x) = 2 5 — Pe, ’ Te W, (6.4) 


VEVP,r 
where w = 1 + 0.01- (i — 1). 


The external disturbance is given by step-wise load jumps at the three nodes 1, 28, 35 from 
different zones in the network as listed in Table 6.3 and illustrated in Fig. 6.8. In order to 
investigate the path dependence of the resulting solutions, the load jumps occurring at t1, t2 
and t3 are compensated by reversed load jumps at t4, t5, and tg (in descending order with 
respect to the order of occurence). 
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Figure 6.8: Step-wise jumps in power consumption (py, qg) for Case Study II. The thin gray lines indicate the 
(constant) power consumption at all other nodes. 


6.3.2 Numerical Results 


Triggered by the step-wise load jumps in p, and qp, respectively, all voltage magnitudes, 
active power generations, and nodal prices attain a new steady state during each of the 15 min 
intervals (see Fig. 6.9). Notably, the jumps at tı and t; as well as those at tz and ts yield 
exactly the same steady state for each of the quantities shown in Fig. 6.9. All nodal frequencies 
converge to the nominal frequency 50 Hz within 10 min and exhibit a maximum deviation 
of —38 mHz after the step at tg. These overshoots of nodal frequencies tend to be higher in 
amplitude if the load jump is closer to the respective node. E.g. after tı, the load jump at 
node 1 in cell 1 causes a much more significant transient frequency deviation for the nodes in 
cell 1 than for those in the other cells. Furthermore, all specified voltage limits are maintained 
during the entire simulation. 


Similarly to Case Study I, the nodal prices X; show a very fast convergence of less than one 
second towards a zonal price A,.. At steady state, the cell-specific active power generations are 
equidistant to each other, while their absolute values depend on all current active and reactive 
power consumptions. These results indicate that active power sharing (cf. Definition 3.38) is 
given across the entire network. whereby the weighting factors w; in (6.4) are identical to 
those in (3.116). 


Congestion Management 


Fig. 6.10 shows the resulting apparent power flows over all boundary lines m € se It can be 
clearly seen that in both scenarios all power flow limits S7"** are respected at steady state. 
In Scenario I-a) (see Fig. 6.10a), there is no transient violation of any of the line flow limits. 
The participation factors & synchronize within 2 min after initialization to a common value 
of less than 0.87 (see Fig. 6.11a). Subsequently, with the apparent power of line 38 — 48 
approaching its specified maximum absolute value, the participation factors split over a range 
of 0.1, where the zonal pricing factor in cell 7 takes the highest value and the zonal pricing 
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Figure 6.9: Nodal frequencies, voltage magnitudes, active power generation, and nodal prices. 
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Figure 6.11: Participation factors K. 


factors in cells 8 and 10 take the lowest value. After the second jump at ta, the line between 
node 35 in cell 2 and node 36 in cell 4 reaches its limit of S32°%3, = 0.2. Subsequently, K2 
increases significantly, while x4 decreases. After the third jump at t3, also the power line 
between node 21 in cell 10 and node 22 in cell 8 reaches its limit. At this point «xə slightly 
recedes, while 19 increases. After the occurence of the opposite jumps at t4 and ts, the 
former steady states are recovered, before all participation factors & converge to their initial 


value 0.87 after the last jump at te. 


Fig. 6.11a reveals that in Scenario I-a), the individual participation factors are overlaid by a 
ripple. By contrast, the individual participation factors in Scenario II-b) resulting from the 
congestion controller with integral action move more slowly and show almost no ripple during 
the transient process, but exhibit higher transient overshoots (see Fig. 6.11b). Accordingly, 
the resulting apparent power flows in Fig. 6.10b exhibit temporary constraint violations of a 
maximum duration of 10 min. Remarkably, the steady-state apparent power flows S3s-,48, 
535-436, and S2122 of Scenario II-b) are exactly at their specified maximum limits, while in 
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Figure 6.12: BoP containment and restoration. 


Scenario Il-a) they only approach these respective limits. This can be elucidated by the fact 
that the barrier functions Um used for the zonal pricing controller for congestion management 
(4.14) without integral action do not allow for an exact convergence to the threshold Ci” 
(and thus |Sm| = S7**), while for the congestion controller (4.15) with integral action, the 


barrier function Um only requires that the integrated constraint violation has to be finite. 


Balancing of Payments 


The trajectories of o indicating the BoP error are depicted in the upper panels of Fig. 6.12. In 
Scenario Il-a), o settles to a new steady-state value between —1.18 and —1.43 after each load 
jump. The resulting bias of o depends on the magnitude of the applied load jump, although 
there is no clear relationship between the sign of the load jump and its impact on øo. However, 
BoP containment is given, which is shown by the bottom panel in Figure 6.12a. Although the 
associated time derived BoP deviations ġ (see ibid.) never exceed 0.016, their accumulation 
over a period of several minutes always causes a net deficit of more than 1. Thus, with respect 
to the sign convention in (4.34), we see that the overall payments of RPs exceed the WPs’ 
overall revenues, thereby resulting in an unwanted accumulation of capital. By contrast, 
in Scenario II-b), ø recovers the desired steady-state value of o* = 0 no later than 15 min 
after each load jump, thus BoP restoration (and hence of course also BoP containment) is 
accomplished. 
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Figure 6.13: Learning-based optimal controller for BoP restoration in Scenario II-b). 


Fig. 6.13 shows the progression of the Lagrange multiplier Y(t) of the employed adaptive 
optimal controller from Chapter 5 as well as the resulting weighting factors w(t) of the 
adaptation procedure (5.55c). The weighting factors wı,...,wıo associated to the basis 
functions =) = (ef)? to E10 = (efo)? converge to the steady-state value wf = --- = wi, = 
0.964 within 5s after starting the simulation, and w11 belonging to =,; = 0? converges to 
wy, = 9.4999. The weighting factors associated to the remaining basis functions converge 
to zero. After the adaptation is accomplished, the value of w(t) is no longer affected by 
subsequent load jump. Yet, at the same time, Y(t) is slightly affected by each load jump, 
while always converging to a new steady-state value within 10s which stays in a corridor 
between 0.978 and 0.985. These results indicate that the underlying HJB equation (5.24) 
is approximately (but not exactly) fulfilled and thus the learned, extended CLF V(a) = 
H,(x,) + w*ZE(x,) is a good approximation of the value function. Occasionally, after Y(t) 
has reached a steady state, the graph of T(t) reveals steep-flanked spikes of Y(t) with a 
magnitude up to 0.12 from time to time. This is caused by a near-zero denominator in (5.55b), 
leading to a division by (almost) zero. 


Zonal Pricing 


Fig. 6.14 compares the resulting (cell-specific) wholesale prices A(t) and the (cell-independent) 
retail price Ar(t). Similar to Case Study I, nodal prices A in each cell k € Z converge to 
a cell-specific wholesale price A, within 1s, such that no differences between nodes can 
be observed in Fig. 6.14. The retail price increases with increasing total demand at the £ 
nodes, and vice versa. In line with the trajectories of «(t) in Fig. 6.11b, there are identical 
wholesale prices, if none of the power flows along the boundary lines reaches its limit, and 
different wholesale prices, if at least one of the power lines reaches its limit (see Fig. 6.10). 
Interestingly, the retail price is hardly affected in cases where A = col,ez{Ax} is after a line 
flow limit has been reached, such as after 19 min through the congestion of line 38 — 48 
or after 37 min triggered by line 35 — 36. Most of the time, each of the wholesale prices 
is below the retail price. This is caused by the BoP containment, which enforces the RPs to 
collectively compensate for the momentary nonzero resistive losses in addition to their own 
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Figure 6.14: Comparison between wholesale prices A and retail price Ar. 
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Figure 6.15: Apparent power flows and corresponding wholesale and retail prices in case of relaxed power flow 
limits for Scenario I-a). 


power consumption. However, after the load jumps at t3 and ts, the spread between wholesale 
prices is high enough that wholesale price A» of cell 2 even exceeds the retail price. This can 
be explained by the fact that cell 2 is located directly at the critical line 35 — 36, thus the 
zonal pricing generates a heavy increase in %2 (and hence A») in order to combat congestion 
locally by means of incentivizing increased infeed by cell 2. 


For comparison, Fig. 6.15 shows the resulting apparent power flows and price curves in case the 
congestion controller of Scenario II-b) is deactivated. In this case, there is a significant overload 
of the lines 38 — 48, 35 — 36, and 21 — 22 (see Fig. 6.15a) by up to 18%. Accordingly, all 
wholesale prices are equal and always lie below the retail price, as can be seen in Fig. 6.15b. 


6.3.3 Summary of Case Study II 


The zonal pricing controller used for congestion management in Case Study II causes a demand- 
oriented, incentive-driven shift of power generation to regions that are “more favorable” from 
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a global grid perspective, whenever the purely merit-order-based network operation (with 
uniform prices) is no longer feasible. Compared to the alternative mechanism of cost-based 
redispatch, however, two major improvements are evident here: First of all, congestion is 
not alleviated “unidimensionally” by simply affecting two participants pairwise. Instead, 
a convenient and automatic adjustment of all cells is executed, as can be seen e.g. in Fig. 
6.11 after the jump at tı. Secondly, the presented congestion controller is able to handle 
multiple congested lines at the same time, as e.g. demonstrated in Fig. 6.10 after the second 
and third jump. By contrast, cost-based redispatch strategies quickly become challenging in 
case of multiple congested lines due to the possibly very large number of combinations of 
increase/decrease pairs. 


Case Study II also shows that the selected pricing strategy has only a minor influence on the 
retail prices, as can be seen by comparison between Az in Fig. 6.14 and Ag in Fig. 6.15b. The 
reason for this is the appropriate settlement of regularized wholesale and retail prices through 
distributed balancing. In combination with the pricing strategy, it provides for additional 
competition between WPs, since, loosely speaking, an increasing «, in one cell is automatically 
compensated by a decreasing x, in some other cell. From this perspective, the automatic 
settlement of zonal price differences resulting from congested lines can be interpreted as a 
dynamic rearrangement of the (copper plate-based) merit order, triggered by network utility 
considerations. 


While the presented schemes for frequency and voltage control, congestion management, 
and BoP containment are fully distributed, the developed adaptive optimal controller for BoP 
restoration is still operated in a centralized manner, since both the complete system dynamics 
as well as all inputs have to be known. Especially, if the state vector of the underlying system 
is very large, this considerably hinders the determination of suitable basis functions. Although 
the trajectories of Y (t) and w(t) can be employed to qualitatively evaluate whether or not the 
value function candidate V(x, w) obtained with a given set of basis functions is a sufficiently 
good approximation of the value function, so far, there exists no constructive approach how 
to reasonably choose the basis functions. Therefore, further research is advisable on how to 
usefully incorporate existing a priori model knowledge into the choice of basis functions. 


6.4 Discussion 


In this chapter, the main capabilities of the developed dynamic zonal pricing controller with 
combined frequency and voltage regulation have been systematically investigated using an 
IEEE benchmark model. Table 6.4 summarizes the respective control objectives substantiated 
in each of the simulation scenarios. Case Study I in Subsection 6.2 showcases the benefits of cell- 
based approaches for frequency and voltage control of large-scale systems. This is in addition 
to Example 4, which already demonstrates the integrated price-based frequency controller in 
its main features incorporating the classical primary, secondary, and tertiary control layers. 
Finally, Case Study II in Subsection 6.3 demonstrates the effectiveness of incentive-based 
dynamic pricing using participation factors for real-time control of line congestion. This 
unification of the classical aspects of power system stabilization and incentive-compatible, 
real-time dispatch represents a new, holistic framework for assigning a concrete, monetary 
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Table 6.4: Scenario chart for Case Studies I and II based on and in supplement to the literature comparison chart in 
Table 2.2. For abbreviations not listed, see ibid. 
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Table 6.5: Comparison of simulation times Topu. 
Scenario Tsim Tcpu 


I-a) 1800 278.65 
I-b) 1800 102.40 
I-c) 1800 67.86 
I-d) 1800 70.72 
II-a) 6300 73.95 
H-b) 6300 389.59 


value to grid support in addition to a pure remuneration for power provision itself—something 
that has not been encountered in the literature so far. 


Table 6.5 lists the computation times Tcpu required for each of the simulated scenarios. It is 
evident that all computation times are far below the simulated time Tsim in each case, which 
shows the real-time capability of the overall control scheme. The individual durations shown 
in Table 6.5 along with further observations from the accompanying simulation experiments 
suggest that the simulation time is highly correlated with the number of active constraints. This 
correlation is plausible taking into account that the NDSolve routine, which is specialized for 
solving differential-algebraic equations, is a variable step solver which tends to a reduction of 
the step sizes (and consequently an increased computation time tcpu) whenever the simulated 
system becomes more stressed due to higher fluctuations of the state variables close to the 
active constraints. However, since the proposed control framework is distributed, in real-time 
applications, separate controllers with significantly lower computational complexity along 
with fixed-step solvers will be operated in parallel at every single node, thereby resulting in 
significantly lower computation times. 


7 Conclusion and Outlook 


7.1 Conclusion 


Modern power systems exhibit a remarkable paradox: On one hand, increasing RES-based 
intermittent power generation requires close cooperation among all resources for efficiency 
and synergy reasons, in order to incorporate a “physical awareness” among all network 
participants about what actions are deemed most ideal from a global network perspective. On 
the other hand, the unbundling principle along with antitrust considerations impose, for good 
reason, that the exchange of information both between system operators and competitive 
prosumers, as well as between prosumers themselves, should be kept to a minimum. 


The dissertation at hand provides a contribution towards bridging this gap by proposing 
an incentive-compatible, market-integrated control scheme with cell-based price consensus. 
Relying on and complying with the classical division into regulated system operators and 
profit-driven prosumers, the proposed brownfield framework allows each self-serving WP to 
autonomously make decisions in order to maximize its profit. In contrast to a copper plate-like 
market which does not consider the location of buyers and sellers [RS19], in the proposed 
framework any information about the temporal and spatial network state is indicated by the 
node distinct real-time price set by the local CCs which act as the network’s neutral market 
facilitators. Inefficient allocations and normative add-on mechanisms such as cost-based 
redispatch or the BRP role, which are both critically discussed in Subsection 2.1.2, can thus be 
omitted. 


Since all power system stability issues and physical constraints are already settled at market 
clearing, there is no further need of a capacity-based BM as compared to the baseline scenario. 
Thus, potential strategic market behavior as discussed in Subsection 2.1.2 is prevented. Conse- 
quently, there are no after-market costs to charge to the RPs’*. In addition to the proposed 
real-time mechanism, the WEM may still be in place, which all network participants may 
use for their own risk hedging or for long-term agreements (cf. Fig. 3.7). If the proposed 
framework is adopted in practice, the only change for market participants is the way in which 
their own real-time prices for electricity generation or consumption have to be calculated. 


For each of the individual network participants and for each of the technical and economic 
layers in Fig. 4.7, this dissertation has derived dedicated control schemes, with a special focus 
on optimization-based controller design and distributed controller communication. The major 
contributions of this dissertation and their relevance to the research gaps addressed in Section 
2.3 are depicted in Fig. 7.1 and briefly summarized below. 


74 Remaining grid tariffs, if any, now only apply for construction and maintenance of the grid infrastructure itself. 
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Research Gap 1 Contribution 1 Theorem 3.44 


Theorem 3.51 


Contribution 2 Theorem 3.54 


Research Gap 2 Theorem 4.22 


Theorem 5.24 


Contribution 3 


Theorem 5.53 
Theorem 4.20 
Theorem 4.23 


Research Gap 3 Contribution 4 


Theorem 4.25 


Figure 7.1: Overview of main contributions. 


In Chapter 3, we have worked towards a market-based integration of power system stability 
by developing a distributed, optimization-based frequency and voltage controller such that the 
actions ofthe local WPs are stimulated solely by their individual nodal price. By thoroughly 
accounting for network parameters in the CCs’ calculations, the price calculated by CCs also 
covers all accompanying line losses, thereby closing Research Gap 1. A sufficient condition 
for stability of the closed-loop system has been provided in Theorem 3.44. 


The rigorous pursuit of an economically efficient interplay between stakeholders has consti- 
tuted a major focus of all endeavors. By introducing a novel, vector-valued zonal participation 
factor K along with a distributed price consensus protocol, the equilibrium set of the resulting 
closed-loop system is restricted to the set of Pareto efficient allocations, which has been 
elaborated in Theorem 3.51 and Theorem 3.54. The dynamic regularization of all participa- 
tion factors by means of distributed balancing, which has been discussed in Theorem 4.22, 
thereby forces only those solutions that provide BoP containment, turning the entire power 
infrastructure into a monetarily self-contained system without any accumulation of capital. 


The methodological foundation for efficiently regulating the remaining steady-state deviations 
of aggregate net payments has been laid by introducing a new adaptive framework for 
optimal control of (generic) PHSs in Theorem 5.24. Since the class of PHSs has proven to 
be a universally applicable modeling tool for a variety of multi-physics or techno-economic 
systems, possible applications of the developed method reach far beyond the particular area 
of power system control applications presented earlier in this work. In Theorem 5.53, the 
developed methodology has been further extended to the case of N-player noncooperative 
differential games. 


Thanks to distributed balancing via the time-dependent regularization factor a, all of the 
remaining |Z| — 1 degrees of freedom when choosing « could be exploited to implement 
higher-level goals. Namely, control laws with and without integral action have been proposed 
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in the context of this dissertation on how to usefully impose “incentive-signaling” spatial 
differences among zonal prices, while also respecting the obligation of treating all RPs equally 
(cf. Claim 4.5). Theoretical conclusions about the resulting closed-loop system have been 
stated in Theorems 4.20, 4.23 and 4.25, and the practicality of the overall control framework 
in automatic congestion control has been demonstrated in Case Study II of Chapter 6, finally 
closing Research Gap 3. 


Since all individual controllers are systematically derived from optimization problems, there 
are no remaining degrees of freedom in the controller design except for the (inverse) controller 
gains T(.), which, however, do not affect the value of the closed-loop equilibrium. Reasonably, 
the controller gains can be used to provide the necessary timescale separation between 
the individual control layers. The overall controller thus features low efforts for parameter 
tuning. 


By automatic control of the participation factors & based on continuous measurements of the 
state of the network, this work has given rise to allocating marginal production in specific 
regions of the power system in real time. In contrast to a pure merit order system, the 
cell-specific multiplication by x provides a real-time incentive mechanism that encourages 
profit-driven network participants with power sources and flexible loads to strategically shift 
their capacities to regions with higher average prices and thus to enhance the reliability of 
the system by inducing an additional geographical awareness among WPs. 


As discussed in Subsection 2.1.2, all disruptions caused by the increasing RES infeed have so 
far been charged to the end customer in an intransparent manner via ever-increasing grid 
tariffs. Through the concept of spatially differentiated prices, which cluster around the (virtual) 
uniform WP price A°, the presented work marks a paradigm shift towards a “polluter-pays” 
principle. Exemplarily shown in Case Study II, where the social optimum gained by uniform 
pricing would fail to alleviate congestion at market clearing, WPs in unfavorable locations are 
subject to significantly lower prices through the dynamic pricing mechanism, which in turn 
pushes them out of the market in the long run. This creates an ongoing market consolidation 
that is favorable from an overall perspective, along with preferential investment in regions 
that are most desirable by virtue of the global pricing policy. 


Remarkably, the proposed control framework does not require any internal model of the WPs 
involved (apart from the minor assumption of rationally chosen cost functions). This not 
only protects the privacy of the stakeholders involved, but also enables the integration of 
an arbitrary number of small-scale flexible prosumers, fully in line with the future vision of 
power networks as transactive energy internet (EI) platforms with minimum entry barriers 
[MKR* 20, PCK* 21]. 


7.2 Outlook 


This work advocates for a careful separation between sovereignty over the grid (held by 
CCs) and sovereignty over electricity generation (held by WPs) to maximize fairness without 
restricting the benefits of a free market. Nonetheless, there will always be the need for a 
certain normative framework set by policy makers, e.g. rules that ensure compliance with 
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Assumption 3.17 or the resolution of overriding long-term goals, in order to keep the results 
of the market mechanism within the bounds of what is socially desirable [NBM* 20]. 


A cell-based infrastructure without cutting the physical network has proven to be advanta- 
geous compared to the MG concept, as it unites the merits of informational and operational 
unbundling with those of non-discriminatory power flows. However, the question of most 
effective cell size and appropriate definition of cell boundaries, which is still open and contro- 
versially discussed in academia, is not explicitly addressed in the context of this dissertation. 
On one hand, as it becomes apparent from the application to congestion control, a finer 
cellularization, especially along lines that may be vulnerable in terms of congestion, basically 
contributes to a more elaborate influence of specific line flows. On the other hand, a too fine 
cellularization counteracts the “no market power” imperative (cf. Assumption 3.17), as it opens 
up unwanted exploitation opportunities for WPs located at critical points in the network to 
affect prices unilaterally, leading to reduced competition in the overall network. Hence, an 
“optimal” cell partitioning must always trade-off between both requirements [Sto97]. While 
in this dissertation, the determination of appropriate price zones is done manually, based on 
a priori knowledge and in a fit-and-forget manner, recent lines of research advocate dynamic 
cell partitioning that, in contrast to static approaches, adaptively adjusts to the instantaneous 
needs of the network [XZK* 16, p. 135ff.; Wan15, p. 97]. In [HDA21], a first practical algorithm 
is presented where the dynamic formation of local energy markets is adjusted on a daily basis 
to gather and reward the flexibility of end users. 


The explicit control methods derived in this dissertation can all be applied regardless of 
the eventual (static or dynamic) cell partitioning and therefore constitute a practical and 
easily implementable toolbox for any superordinate cellularization and pricing strategy to 
conveniently realize the intended higher-level goals. 


The explicit concepts for real-time pricing elaborated in this dissertation open up a broad field 
for extensions and amendments. For instance, by adequately extending K” (see Subsection 
4.3.3), additional summands could not only incorporate generic long-term goals, but also 
account for network effects anticipated for the future right before their actual occurrence. This 
could contribute to an internalization of follow-up costs as current costs and therefore pave the 
way towards a proactive energy infrastructure [MS21]. In particular, the distributed balancing 
mechanism from Subsection 4.4.1, if active, always ensures that any of these additional price 
components which incorporate future tendencies are already fairly allocated among consumers 
as of today. 


Especially for large-scale power infrastructures, aiming for a distributed controller implemen- 
tation with neighbor-to-neighbor communication, as pursued in this work, is advantageous 
both in terms of computational complexity and cyber security. However, compared to the 
other control schemes presented in this dissertation, the novel adaptive methodology for 
optimal control of single- and multi-player PHSs introduced in Chapter 5 still requires a 
complete model of the physical plant system (along with perfect measurement information on 
the other players’ control actions in the multi-player case) and therefore lacks a distributed 
implementation. Paving the way for the application to large-scale systems, future research 
should thus focus on optimal controllers and noncooperative differential games for PHSs with 
limited information sharing and non-exact model knowledge. In addition, further emphasis 
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could be put on more sophisticated adaptation strategies for the individual weighting factors 
to increase the speed of convergence while still guaranteeing asymptotic stability. 


The major stumbling block towards a comprehensive stability analysis of the overall system 
dynamics is the general absence of Lyapunov-based a priori statements in case of lossy power 
networks (see also the discussion in Subsection 3.4.4 on ongoing research efforts). For this 
reason, the existing literature on PHS-based control of AC power systems is almost without 
exception confined to the lossless case. In the context of this work, a sufficient condition 
for stability has been presented by Theorem 4.25, which, however, contains an impractical 
criterion that requires a priori knowledge of the numerical steady-state value. This in turn 
requires the solution of an intractable (and possibly high-dimensional) nonlinear system of 
power flow equations, which in general is only given through approximation (cf. [Tay15]). 
A trend in recent research on power system stability and control is therefore towards the 
design of versatile controllers that preserve equilibrium-independent passivity of the overall 
system [SNM*21]. Future advances in this field may allow to derive controllers whose 
stabilizing properties do not depend upon a condition that is difficult to prove in advance— 
yet at the price of a conservative controller design with limited performance and/or the 
abandonment of potential degrees of freedom. 


Another requirement to be critically investigated by further research is the dynamic security 
requirement of “sufficient timescale separation” (cf. Section 4.5), which was necessary in 
the context of this work, but is generally difficult to certify. For this purpose, an even more 
rigorous first-principles modeling e.g. by means of a thorough port-Hamiltonian modeling 
of the physical plant system and all controller parts bears great potential towards a more 
native understanding of the (generalized) power flows of the individual subsystems along 
with their particular impact on overall system stability. Such a generalized stability analysis 
could eventually obviate the need for any artificial security assumptions regarding timescale 
separation. 


Yet the goals of the energy transition can only be achieved if all parties are enabled in equal 
measure to be seen as part of the solution. Due to its variability over time, the retail price Ar 
already introduced as part of this dissertation provides a thriving linkage point for research 
and innovation on demand-side oriented participation. Suitable price incentives and low-entry 
optimization tools for personal use might thus also encourage RPs to play a much more 
active role in future electricity markets. Potential approaches currently being piloted range 
from smart micro-storages at household level [MGR* 18], widespread installation of every 
household with a smart meter infrastructure [BM17, CGM19, YLGW19], and app-based optimal 
scheduling to exploit smart home flexibilities [ATM* 20], to the deployment of completely 
decentralized EI platforms for automated settlement of smart contracts among micro-scale 
energy resources, e.g. on the Ethereum blockchain [ACRK19, BAA™ 21, SKSJ21]. However, 
two factors are decisive to the success of such concepts: First, there must be a political debate 
about how much flexibility is tolerable for the individual and how much individual inflexibility 
is tolerable for society. Secondly, any innovation that enables the participation of RPs must 
be made accessible to everybody with lowest possible entry barriers. If unbureaucratic ways 
to involve end users can be enabled on a large scale and kept accessible for everyone, there 
is huge potential in turning the narrative of energy transition from a bare necessity to a 
collaborative societal project where everyone willing can participate and benefit. 


A Mathematical Supplements 


A.1 Notation 


Throughout this dissertation, vectors and matrices are written in boldface. All vectors are 
column vectors a = col;{a;} = colfaı,aa,...} = (a1,@2,...)' with elements a;, i = 
1,2,.... The notation A = diag{a} = diag, {a;} means that A is a diagonal matrix with 
the entries of vector a on its diagonal. The all-zeros and all-ones vectors of dimension n 
are denoted by O, and 1, respectively. The (n x n)-identity matrix is denoted by I„. The 
unit vector of the k-th axis of a Cartesian coordinate system is denoted by ex. For vectors 
a,b of the same size, a > b denotes that each component in a is greater than or equal 
to the corresponding component in b. Positive semidefinite and positive definite matrices 
or functions are denoted by = 0 and > 0, respectively. Moreover, we write V(x) > 0 if 


V(x #0) > Oand V(0) = 0, and V(x) = 0 if V (x £ 0) > Oand V(O) = 0. 


Optimizers or equilibria are marked with a star L1* and shifted values with respect to the 


optimizer or equilibrium are marked with a tilde Õ, i.e. &(t) = x(t) — x*. Upper and lower 


bounds of a variable are denoted by O and L, respectively. Phasor variables are marked with 


an arrow [and the complex conjugate of a variable is marked with an asterisk 


To allow distinction from the Hamiltonian H of the input-state-output PHS (cf. Defini- 
tion A.13), the Hamiltonian function of the optimization problem (also called Pontryagin 
function) is denoted by #. With the set of players P, the vector u~; refers to the vector of all 
inputs j € P\{i}. Where appropriate, we also the subscript index ~i to denote all elements 
from a given index set except 1, if the index set becomes clear from context. For clarity of 
presentation, the time dependence (t) of the variables is not explicitly mentioned, unless it is 
essential for transparency of the statements. If no dimension is specified for a certain vector 
or matrix, it can be derived from context. 


For scalars a,b € R, we define 


a, a>Oorb>0, 
(a), = | (A.1) 


0, otherwise, 


and for vectors a,b € R”, (A.1) is applied component-wise, i.e. (a)} = col{ (a1) 5. or 


Coane }. For a scalar a € R, we define the saturation operator 


1, a>l, 
(a)-;=4 a, a€ [-1,1], (A.2) 
-1, a<-l, 
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and for vectors a € R”, (A.2) is applied component wise, i.e. (a)! , = col((a1)1,,...,(an)+4). 


The operations absolute value | - |, natural logarithm In(-), and exponential function exp(-) 
when applied to vectors a € R” are applied component wise, e.g. |a| = colllaı|,... , |an|} 
etc. The Hadamard product of two vectors is denoted by o, i.e. a o b = col{a1b1,. . . , anbn} 
for each a,b € R”. 


The gradient of a function f: R” — R with respect to x is denoted by V f(x) and is the 


column vector of all partial derivatives of f(x), ie. Vf(x) = col{ te) ee ope) }. The 


Jacobian of ® : R” — R” with x > (x) is defined as oe = (V (£), ..., V®,(x))". 


A.2 Basic Concepts of Game Theory 


The conceptual framework for the investigation of decision-making situations among multiple 
network participants with different, possibly conflicting goals is provided by game theory. In 
recent decades, game theory has been established as a lingua franca for almost all research 
on competitive energy markets [ANHMIT20, p. 86; NBM*20, p. 2], since it not only offers 
modeling tools for conflicting situations, but also explicit instructions for strategic actions of 
the participants as well as a normative assessment of desired and undesired operating points 
from participants’ viewpoints. After first conceptual studies in the 1970s [MD77, BHK78], 
game-theoretic concepts modeling the interaction of competitive network participants have 
been developed at every timescale, from optimal day-ahead scheduling on an hourly basis 
[AOS*13b] to real-time strategies using differential games” [LLL*19]. Moreover, a wide 
variety of use cases and network participants have been considered, from optimal generator 
dispatch, demand-side management and demand response” [MWY* 19], to optimal balancing 
of interconnected distribution systems and optimal utilization of shared resources [MMM20]. 
An early review on the applications of game theory for power systems is conducted in 
[SHPB12]. For more recent survey papers, see [CY19, ANHMIT20, NBM? 20]. 


In the context of game theory, each network participant is modeled as element 7 € P of a set 
of players. It aims at maximizing its own profit P; by choosing a strategy aj. 


Definition A.1 (Game [BO99; Bau 16, p. 4f.]). A game is a tuple (P, (Aj) iep, (Pi)ier), 
where P = {1,..., N} is the set of players, A; is the set of actions of player i € P, and 


P;: A, x---x An OR (A.3) 


is the profit function of player i € P. 


For convenience of notation, let A = A, x --- x Ay denote the set of action profiles, where 
an action profile a € A is the collection of all actions a; of the players. If P; depends not 


75 (cf. Definition A.2 in Appendix A.2) 
76 The term “demand-side management” encompasses all (long- and short-term) control actions at consumer side, 
while “demand response” specifically refers to the provision of short-term incentives for price-elastic consumers. 
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only on the actions a; € A; but also on the state x(t) of a dynamical system, this leads to the 
notion of a differential game: 


Definition A.2 (Differential Game [SH69; BO99; Bau16, p. 90]). A differential game 
is a tuple (P,5, (Ax)ier, (Pi)ier), where P = {1,..., N} is the set of players, $ is a 


controlled dynamical system 


a(t) = Fett), u(t), (A.4a) 
(0) = £o, (A.4b) 


where f : R” x RP! x... x RPN is Lipschitz continuous with state x € X C R” and 
inputs u; E€ U; C RP‘. A, is the set of actions of player i € P with A; =U,, and 


P,:AxXY OR (A.5) 


is the profit function of player i € P. 


Remark A.3. Independent to the notion of differential games, the term dynamic game encom- 
passes those games where the players make their decisions A, sequentially, in contrast to static 
games, where players act simultaneously. © 


A common assumption in classical game theory is that all players are subject to the following 
basic behavioral pattern: 


Assumption A.4 (Rationality and Common Knowledge of Rationality [Aum76, 
AB95, Bic04]). Each player acts rationally, i.e. strives to maximize its own profit P;. 
Moreover, each player takes into account that the other players act rationally as well. 


An important implication of Assumption A.4 is that especially those strategies are preferable, 
where no player can achieve a higher profit without at least one other player achieving a lower 
profit. Such strategies are said to be Pareto efficient or Pareto optimal. A formal definition is 
given below: 


Definition A.5 (Pareto Efficiency [Bau16]). The action profilea* = (aj,..., až 
ax,)'ransp is said to be Pareto efficient if there exists no other a € A such that 


dep: P;(a) >P;(a*) A P_i(a) > P-;(a*). (A.6) 


If, instead, there exists ana € A fulfilling (A.6), then a* is said to be a Pareto inefficient 
or Pareto dominated strategy. 


For existence theorems on Pareto efficient solutions, see e.g. [Meh85]. 
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Another prominent solution strategy arising from Assumption A.4 is to find Nash equilibria 
among all possible action profiles a € A: 


Definition A.6 (Nash Equilibrium [Bau16, p. 8]). The action profilea* = (a},...,a},) 
is said to be a Nash equilibrium, if none of the players can make a higher profit by unilat- 
erally deviating from its own action profile, i.e., if 


Va; € A;: Pi (ai, a_i) > P;(a}, a* ) (A.7) 


=; 


holds for alli € P. 


Depending on the information structure of the given differential game, we distinguish between 
open-loop strategies, where each player only has knowledge of the initial state £o of (A.4), and 
closed-loop strategies, where each player is able to observe the actual state x(t) of (A.4). 


If all profit functions P; are Lagrange-type, implicit solutions exists for both open-loop and 


closed-loop Nash strategies. 


Definition A.7 (Lagrange-Type Performance Index). A profit function is said to be a 
Lagrange-type performance index, if it has the following form. 


-f E AO (A.8) 


In the open-loop case, the Nash strategy of each player is determined by solving a boundary 
value problem which follows from Pontryagin’s maximum principle [Bre11]. The resulting 
open-loop control laws depends only on time, thus a disturbance leads most likely to subopti- 
mal operation. 


In the context offeedback control, where the actors base their strategic decisions on continuous 
measurements of the state vector and have limited information about other actors’ objectives, 
emphasis is placed on finding closed-loop Nash strategies (also called feedback Nash strategies). 
Here, the players are able to react to disturbances by immediately noticing undesired deviations 
from the desired state. An implicit solution is given by a set of coupled Hamilton-Jacobi- 
Bellman (HJB) equations: 


Lemma A.8 (Sufficient Condition for Feedback Nash Equilibrium [Bre11, HKZ12]). 
For the differential game from Definition A.2 with fixed duration [0, tr], the action profile 


* * 


a* = (uX(t),...,ui,(t))' provides a feedback Nash equilibrium, if there exist continu- 
ously differentiable value functions V,* : [0,T] x X — R for each i € P, such that the 
HB (partial differential) equation 


x x T 
OVE) = max >) f(t, £, wi, umi) — TE (A.9) 
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and the terminal condition 


Vý (tæ) = 0 (A.10) 


are satisfied for all (t, x) € [0, T] x X. 


Note that for N = 1, the above differential game with Lagrange-type performance index is 
equivalent to the classical optimal control problem. For existence theorems for open-loop and 
feedback Nash equilibria, see [Bre11, p. 360ff.]. 


A.3 Algebraic Graph Theory 


Definition A.9 (Graph, Directed Graph, Connected Graph, Complete Graph 
[Bap14, p. 10; HKK* 05, p. 9; BM76, p. 4]). A graph % is a pair (V, E), where V is the 
set of nodes and E is the set of edges. The graph is said to be directed if the edges e € E are 
ordered pairs e = (v1, v2), where vı € V represents the tail of e and va € V represents the 
head of e. The graph is said to be (weakly) connected, if there exists an undirected path 
connecting every pair of nodes in V. The graph is said to be complete, if there exists an 
edge between each pair of nodes. 


Definition A.10 (Incidence Matrix [Bap14, p. 13]). The incidence matrix D € R”*™ 
of the graph G is defined by 


1, i is the head of ej, 
—1, iis the tail of ej, (A.11) 


0, otherwise. 


IfG is a weighted graph, then the non-zero entries in (A.11) are multiplied by the weights 
wij € R of the respective edge (i, j) € E. 


Definition A.11 (Laplacian Matrix [Bap14, p. 49]). The Laplacian matrix A € R”*” 
of the graph G is defined by 


A=DD'. (A.12) 


From Definition A.11 it follows that [A]; ; = [Al;.; = —1, if nodes i and j are incident, and 
[A];,; being equal to the degree of node i € V (ie. the total number of edges connected to node 
i). The incidence and Laplacian matrix of a connected graph have the following properties: 
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Lemma A.12. Ifa graph Y = (V,€) with |V| = n is connected, then the following 
statements hold: 


(1) rank(D) = rank(A) = n — 1, 


(2) the all-ones vector 1 is a basis for the kernel of both D" and A, 


(3) the Laplacian matrix A is positive semidefinite and has exactly one zero eigenvector. 
The remaining eigenvectors of A are strictly positive. 


Proof. (see [Bap14, pp. 14, 50f.]) 


A.4 Port-Hamiltonian Systems 


In recent years, systematic modeling of dynamical multi-physics systems” in port-Hamiltonian 
form has become increasingly popular in a wide range of applications such as acoustics [FH16], 
aerospace [ACRMA19], robotics [GSC17, MMS07, MMS09], power electronics [BDST17, 
CGB* 19], material science [LSM* 20], or energy systems [FZO+ 13, SOA* 14, CT14, HMM* 20]. 
Among the broad class of implicit or explicit PHS representations (see e.g. [CGHM13] for 
a comprehensive overview), the state-space representation via input-state-output PHSs, in 
which the system dynamics are affected by the gradient field of a smooth, positive definite 
Hamiltonian H (a) representing the overall energy stored in the system, is very familiar for 
modeling and control purposes, as it allows direct access to the inputs u € R? and outputs 
y € R? of the system. 


Definition A.13 (Input-State-Output PHS [van17, Definition 6.1.1]). Let 2 € R” 

be a state vector. The state-space system 

OH (x) 
Ox 


è = (J(x) - R(@)) 


H (x) 
Ox 


(A.13a) 


y = G' (x) 


(A.13b) 


is said to be an input-state-output PHS (without feedthrough), if J (x) = —J'(a) and 
R(x) = R'(x) = 0. The gradient z := V H (æ) is said to be the vector of co-states or 
generalized efforts of (A.13). 


The above input-state-output representation clearly visualizes a separation between the 
interconnection topology of the system and the constitutive relations of its components, 
which is an inherent property of PHSs [DBMS09]. The input-state-output representation of 


77 The PHS representation can be applied whenever energy plays a major role in the dynamical system to be 


modeled. Indeed, this modeling paradigm can also be abstracted to non-physical systems, e.g. by conceiving the 
momentary value of the objective function of gradient flow (cf. Subsection 3.4.1) as generalized cyber-energy 
stored in the system [JOv17, p. 109]. 


A.4 Port-Hamiltonian Systems XXXI 


PHSs is thus particularly suitable for modeling of large-scale power systems with a repetitive 
structure. A generalization of input-state-output PHSs is given by the following definition: 


Definition A.14 (Input-State-Output PHS with Nonlinear Resistive Structure 
[van17, Definition 6.1.4]). Let x € R” be a state vector. The state-space system 


(A.14a) 


(A.14b) 


is said to be an input-state-output PHS with nonlinear resistive structure, if J (x) = 


—J' (æ) and z™R(a) > 0. 


Definition A.15 (Passivity, Shifted Passivity [van17, p. 94ff.]). A nonlinear state- 
space system 


t = f(a,u), (A.15a) 
y = g(z,u), (A.15b) 


where f : R” x RP — R” is locally Lipschitz, g : R” x R? — RP is continuous, 
f(0,0) =D and g(0,0) = 0 is said to be passive if there exists a positive semidefinite 
storage function V (æ) such that 


(A.16) 


holds for all (x, u) € R” x RP. 


Let (a*, u*) with f (a*, u*) = O denote an equilibrium of (A.15). Then (A.15) is said to 
be shifted passive with respect to (a*, u*) if there exists a positive semidefinite storage 
function V (a) with 


(A.17) 


The following corollary trivially follows by comparison of Definition A.15 with Definition 
A.13 and Definition A.14. 
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Remark A.17. Depending on the application, different sortings of the state vector x are chosen 
in the context of the dissertation. However, the PHS property is invariant with respect to these 
permutations. Consider e.g. the state transformation x' = Px, where P € R”*” denotes an 
arbitrary permutation matrix. Then it can be directly inferred that the transformed input-state- 
output PHS 


&' = (PJ(«)P!'- PR(a)p-) C2) + PG(ax)u, (A.18a) 
m DB Oa! 
J(x') R(x’) 
u rOH(«) 
y=(PG(e)) 7, (A.18b) 


fulfills J (x') = -J'(x') and R(x') = R! (x') = 0. This property also holds for the general 


case of input-state-output PHSs with nonlinear resistive structure. 


A.5 Convex Optimization 


Definition A.18 (Optimization Problem in Standard Form). 


min folz) 
subject to f(x) <0, 
= 0, 


hj(æ) 


Definition A.19 (Convex Optimization Problem). The optimization problem (A.19) 
is said to be a convex optimization problem if f;(x), i = 0, .. . , p are convex functions 
andh;(z),5 =1,...,m are affine functions. 


Definition A.20 (Karush-Kuhn-Tucker (KKT) Point [BV15, p. 243ff.]). Consider 
the optimization problem (A.19). Each (a*, A”, p*) is said to be a Karush-Kuhn-Tucker 


point, if 


ar, a, p*) =0, (A.20a) 
oe (at A", u") = 0, (A.20b) 
i file") = 0, i=1....,D, (A.200) 
w>0, (A.20d) 
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where 
L(x, A, p) := folæ) + I file) + I Argh; (a) (A.21) 


denotes the Lagrangian of (A.19). 


Definition A.21 (Slater’s Condition [BV15, p. 226f.]). The convex optimization prob- 
lem (A.19) is said to fulfill Slater’s condition, if there exists some x’ which satisfies (A.19b)- 
(A.19c) and f;(x’) < 0 for all non-affine functions fi. 


Lemma A.22. Consider the the convex optimization problem (A.19) and assume that 
Slater’s condition holds. Then x is a global optimizer of (A.19), if and only if (a*, A”, u*) 
is a KKT point. 


Proof. (see e.g. [BV15, p. 226]) 


A.6 Basic Principles of Welfare Economics 


Definition A.23 (Market Equilibrium, Market-Clearing Price, Market-Clearing 
Quantity [Deb59, Yu20]). A market equilibrium is a condition where a market price is 
settled through competition such that the amount of produced goods equals the amount of 
demanded goods. The price reached at market equilibrium is said to be the market-clearing 


price or competitive price, and the quantity reached at market equilibrium is said to be 
the market-clearing quantity or competitive quantity. 


Definition A.24 (Perfect Market, Perfect Competition [Bor93, Ley18, Yu20]). A 
market structure is said to be a perfect market or perfect competition, if the following 


idealized conditions are fulfilled: 
1. The traded goods are completely homogeneous (i.e. indistinguishable). 
The market participants act as price takers. 
All market participants are fully informed about current market conditions. 
The traded goods can be divided arbitrarily. 


There are no transaction costs incurring. 


N OW KF WwW N 


All market participants act infinitely fast. 
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7. There is a large number of suppliers and demanders interacting with each other, so 
that none of them can individually influence market prices. 


A perfect market has the following feature. 


Theorem A.25 (First Fundamental Theorem of Welfare Economics [Moo06, 


p. 150]). In a perfect market, each market equilibrium leads to a Pareto efficient allocation 
of resources. 


In monopolies or oligpolies, individual suppliers have a considerable influence on prices”. 


However, if there are a sufficient number of players (polypoly), the assumption of price-taking 
behavior is justified, as a single player has only negligible influence on the overall price. 


A simple but powerful welfare metric” is the sum of aggregate profits (utilitarian or Ben- 
thamian welfare function) [Var10, p. 635], which in our case equals 


w= > Pr +), Pr), Ch. (A.22) 


TEW kez keZ 


The usefulness of W becomes clear in the following result: 


Theorem A.26 (Pareto Efficiency of Market Equilibria under Perfect Competition 


[Var10, p. 636ff.]). W attains its maximum in an equilibrium of a perfect market. 


Remark A.27. Theorem A.26 states that in a perfect market, the point of social optimum (i.e. 
arg max W) and the point of individual optimum (in the sense of individual Pareto efficiency) 
coincide. From a welfare point of view, it is therefore desirable that the conditions of Definition 
A.24 are satisfied to the best possible extent. © 


78 Oligopolies with few participants are said to be tight, and those with many participants are said to be wide. The 


wider the oligopoly, the closer the market equilibrium is to the social optimum. 

Another welfare measure which is sometimes encountered in the context of energy economics is the maximin (or 
Rawlsian) welfare function [Var10, p. 636] W = min;cp{P;}, where only the profit of the “worst-off” market 
participant is relevant. However, since the profits of the other market participants are completely disregarded and 
furthermore W’ does not conform to the chosen profit functions for the case where WPs have different nominal 
outputs, this type of welfare functions is not considered further. 


79 


B Supplementary Material to 
Optimization-based Control of Cellular 
Power Networks (Chapter 3) 


B.1 Synchronous Machine-Type Nodes: Connection with 
the Power Network 


Fig. B.1 shows a vector diagram of the stator voltages U; and Ug; and their relationship to the 
external nodal voltage U;. For convenience, we define a synchronous rotating reference frame 
and denote the absolute rotor angle with respect to the same reference frame by 6;. p; defines 
the load angle between stator current and nodal voltage®’, Vg ; is the rotor angle between Uri 
and U! ‚and hn ; is the (pseudo-) rotor angle with respect to the synchronous rotating reference 
frame of U;. The current and voltage phasors in synchronous rotating reference frame can 
be characterized by Ü! = Ülexp 7d; = U; exp 7(0; + Uni), De; = Ug exp(7(9i + Vg,i)), 
U; =U; exp(j6;), and I; =I, exp (6; — yi) = I; exp 9(6; + Uni — Pi). From the right half 
of Fig. 3.4, it follows that 


Ü; — Üg i = Xailai + IXa i (B.1) 


Moreover, it follows that Uga,i — Uai = JAÄT,ilai + JXn,ilq,i and Ugg, i — Ugi = 9X Talay + 
jXn,ila,i, hence 


Ü; — Ü; = 9Xailai + Xq alga + (Xt + Xn) (Lai + Ian): (B.2) 
With the geometric relationships from Fig. B.1 
Iai = Iisin(g}), (B.3a) 
Io, = I; cos(y;), (B.3b) 
Uq, = Ui sin(Ûn,i), (B.3c) 
Ua, = U; cos(Vn,i), (B.3d) 
we can calculate the active and reactive power injections 
Pinja = RULES) = Uili cos(ys) = Ui; cos(p} — Yn) 
= U; I; sin(p}) sin(a) + U; I; cos(y;) cos(On i), (B.4) 
ete (ÜI) = = U;I; sin(p;) = Uili sin(p} — Vn.) 
= U; I; sin(p}) cos(d,,;) — U; I; cos(y;) sin(On,i). (B.5) 


80 The SM acts as generator if Vn, > 0 and as motor if On. < 0. 
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Qi __~ d-axis `a d-axis 
7 Ta, 
t 
/ 
ie : 
’q-axis 
(a) Synchronous generator (Vn, > 0) (b) Synchronous motor (Uni; < 0) 
Figure B.1: Vector diagram of a SM. 
Inserting (B.3) into (B.4) and (B.5) yields 
Pinji = Vailas + Ug ila; (B.6) 
linji = Ug ula mUd iaa (B.7) 


Since Tai and I: are orthogonal, (B.2) can be written separately for both d- and q-axis, which 
yields 


Dr = Uqi = Tai(Xai + ATi + Ani) (B.8) 
Uji — Vai = I Kai + X Ti + Xni) (B.9) 


With Assumption 3.7b), we have U j ; = 0. Moreover, we define the auxiliary reactances 
Xan,i = Xai + XTi + Ani and Agia — Aaa + XT j + Ani to get 


_ U-TU;cos(d,,;) 


Tiz 5 B.10 
a Xan,i ( a) 
U; sin(dn i) 
Tog = SS B.10b 
q, Xan,i ( ) 
Substituting (B.10) into (B.4) and (B.7) yields 
U!U; U? Xani = Kan,i 
Pinji = —— sin(dni)t+ < - ~ sin(20, i), (B.11) 
J Xan,i ( l ) 2 Xan, iXan,i ( j 
UU; U? À 
Ginj,i = X, 7 cos(Vn,i) = SETE, (Xan,i sin?(0n,i) + Aani cos*(In,i)) (B.12) 
nti n,i+\qn,i 
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B.2 Distributed Consensus-Based Control 


In the area of networked systems, there often arise control laws of the separable type 
ù=} flu), (B.13) 
i=1 


where the calculation of the global variable u is composed of a finite amount of additive com- 
ponents f;(w) with i = 1,...,n, representing “local” calculations, e.g. relying on parameters 
which are only locally available. For the local calculation of f;(u), however, each local agent 
needs instantaneous knowledge of the global variable u. Additionally, in order to calculate ù, 


all f;(w) need to be known globally. 


The idea of consensus-based control [OSFM07] is to design local controllers for agent i which 
achieve the same equilibrium as (B.13), but without requiring knowledge of u or f-;(u). 
Instead, the agents communicate local variables in a distributed fashion via m neighbor-to- 
neighbor communication links. The communication structure is represented by a weakly 
connected, directed graph with incidence matrix D € R”*”. 


Lemma B.1. Consider the centralized controller (B.13) and let the steady state u* of (B.13) 
be unique. Then, each equilibrium (u*, v*) of the distributed consensus-based controller 


filu) 


Tut = + Dv, (B.14a) 


falun) 
r,v=-D'u (B.14b) 


with Tu, Ty > 0 is unique. Moreover, it holds that u* = u* -1. 


Proof. The equilibrium of (B.14) is characterized by 


Flur) 
0 = : + Dv", (B.15a) 
0=-D'u*. (B.15b) 
Since D is the incidence matrix of a connected graph, (B.15b) implies u] =... = ux =: ü*. 
Premultiplying (B.15a) with 1" yields 
Mi @)] n 
x T * _ ^k 
+EDv* =X fâ"), (B.16) 
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which is equivalent to (B.13), implying that ü* is equal to u*. To prove uniqueness of v*, 
define v =: vj + vg, where vy € im(Dr,') and vg € ker(Dr,,'). We show that both vf 
and vj are unique, which implies that v* is unique as well. 


With regard to vx, note that for each b € ker(D77!) it holds that b'v = 0 and thus 
b' v(t) = b'v(0) for all t > 0. Moreover, with b'v(t) = b' v(t), it follows that vx = 
vx (0), thus v% is unique. With regard to vf, note that dim(im(Dr,,')) = rank(D) since 
T, has full rank. With (B.15a), we state that v7 is the unique solution of 


fi(@) 
— : - Dv% = Dvr. (B.17) 
fn(@) 


This completes the proof. 


In summary, the distributed consensus-based controller (B.14) enables each agent to calculate a 
local estimator of the global variable u, which is communicated among agents in a distributed 
sense by using the communication structure represented by D. 


Remark B.2. If (B.13) has v > 0 equilibria, then the distributed controller (B.14) also has v 
equilibria. This fact can be proven by formulationg the proof of Lemma B.1 separately for each 
equilibrium u}, ..., u% of (B.13). & 


Remark B.3. The results of Lemma B.1 remain the same if the incidence matrix D in (B.14) is 
replaced by an arbitrary Laplacian matrix £ (cf. Definition A.11). © 


B.3 Extension to Nonconvex Cost Functions 


Strict convexity of Cg ; is an assumption that is very common in the literatur for simplification 
and regularity reasons. However, with a closer look, certain types of power plants exhibit a 
more delicate cost structure, whereby the cost function can be divided into three major areas 
which are caused by different levels of efficiency, see solid line in Fig. B.2a: The first area 
is characterized by relatively low efficiency, which is reflected by a rather steep increase of 
Ce i(Pe,i) per increment of power. What follows is a region of high efficiency, which causes a 
flattening of Cg i (Pg,i). At even higher levels of power, it becomes increasingly difficult (that 
is, increasingly expensive) to supply further amounts of power, with the consequence that 
Cg i (Pg, i) eventually rises with increasing steepness. 


Let the (unique) inflection point of C, ;(pg,i) be denoted by pg a Then, also p, ; is an inflection 
poe of P, ;(Pg,;). Now assume that the current price X; fulfills Ai 2 Vey. KA i) and define 


by De := arg max Pg į under the current price \;. Then, it is rational for each profit-oriented 


81 This point corresponds to the operating point of the power plant with highest efficiency. 
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WP to apply a virtual cost function Čs i which is strictly convex around ph in order to 
guarantee global convergence to the own profit maximum. If however A; > VCg,i(p¢.;), it 
is advisable for WPs to leave the market, since otherwise any nonzero power supply would 
result in a negative profit. 


To obtain a continuously differentiable, strictly convex upper envelope of C, ;, it is thus a 
reasonable choice to choose the virtual cost function equal to 


č a Cgi; Pg,i > Poi 
gi — 2 2 iE 
Poi De) 2p? Gee Poi) Chi Pees Pea 


‚i 


(B.18) 


where m° = VC,,(p2 ,) denotes the steepness of C,,;(pz,i) at the inflection point. We 
motivate this choice by a small example. 


Example 9 


Consider the cost function 


1 


Cas Pea) = w (Dei — 2)° + (pei — 2)? + 2(pg i — 2) + 8). (B.19) 


o _ 5 mo _ 27 o_1 œ ; Co. 
Here, Pri = 3 Cad = 50° and m = z- Fig. B.2 shows the curves of Ce; and Cg į as well 


as the resulting profit functions Pg i for different prices A; € [5,25]. It can be seen that the 
replacement of Cg i by the strictly convex, continuously differentiable function 


Č (B.20) 


10 

RN Ce i, Pg,i > 6? 
gi = 10 
6 


2 1 509 
Psi 30Ps,i t Ios? Psi < 


# 


does not affect the respective value of the best response pẹ ;. 


(a) cost function (b) profit functions 


Figure B.2: Plot of the nonconvex cost and profit functions (solid lines) and their respective “virtual” substitutes 
resulting from convexification (dashed lines). Notably, the convexified profit functions exhibit the 
same maxima as their nonconvex counterparts. 


XL B Supplementary Material to ‘Optimization-based Control of Cellular Power Networks’ (Chapter 3) 


B.4 Parameter Values for Example 4 


The units of the parameters are given in p.u., except T4, ; which is given in seconds. 


Table B.1: Numerical values of the nodal parameters used in Example 4. 


Node Type A; By Ti Taos Xai -— Xai 


1 S 1.60 —5.5 5.22 6.45 0.0159 
2 S 1.22 —5.5 3.98 7.68 0.0213 
3 L 1.38 —3.3 0.45 = — 
4 T 142 -3.1 0.42 _ = 
5 I 1.40 —7.0 0.44 = — 
6 L 1.30 2.0 = 
7 L 1.30 2.0 — 


Table B.2: Numerical values of the line parameters used in Example 4. 
Bı2 Bis Bie B23 Bas B34 Bas Bs7 
127 1.4 2.0 14 205 11 1.0 2.0 


C Supplementary Material to Real-Time 
Incentives by Zonal Pricing (Chapter 4) 


C.1 Proof of Theorem 4.25 


Proof. Recall the physical plant system (3.121) and define K := (diag(«))~+. Now consider 
the Lyapunov function candidate 


Ss ~ 1/- = ~T = ET = 
V(x) = H (p) + 5 (Ba) Kr eb, + Me KT yp, Mey the KT yp, Me 


ee A ee pee rt ~ ~T ~ 
+A TAAHU TLV +U; TUU + Rs; Tus,Hs+ + Hs-Tus_Hs- 


—_T > ST, = = = 
+UITU UT + ory T py, Hr + Mt —T pz Ar) , (C.1) 


where H,(&,) = H,(&p) — (5) VH (x5) — H,(x%) is defined as in the proof of Lemma 


3.39. Note that the Hessian of H, (p) at Zp = O is equal to the Hessian of Hp (æp) at £p = £} 


and thus positive definite by Assumption 3.33. Furthermore, since Hp (0) = 0 there exists 
a neighborhood around %p = 0 with Hp (čp) > 0. As all terms in (C.1) except H,(&,) are 


quadratic forms with positive metrics, Hp(®%p) > 0 also implies that V(x) > 0 holds in a 
(possibly larger) neighborhood around x = 0. 
The derivative of V(x) along the trajectories of (3.49), (3.110) equals 
Yon eT py _%5T x sth = 
V(&) = 2, I pZp — 2, (Rp(&p) — Rp(xF)) + 2, Gpup 
— Pe K(VC(p,) — VC(p3)) — Pe e — By K figs 
T Pe K iig- + ji, K (Ps = Pa + ji, K(p, ze Pg) jig 
To sT sT. ~T. 
+A e—U,U; -Ut us, + U; Hs- 
+ üs (Ur -Upis ths- (Us Ur)... 
~T ~ ~T ~T 
—U,Uz—Uzpyz, + Urh- 
+ ùz- (Uz — Ut)h,_ +Mı-(Ur-Un)),,- (C.2) 


Rearrangement of the expressions in (C.2) yields 


V(x) = Wi (x) + Wz(æ) + W3(æ) + Wa (x) + W5 (æ), (C.3) 
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where 
Wi(x) = ZI p%p, (C.4) 
Wo(ae) = -ZI (Rp(@p) — Rp (ws) + 2, (C5) 
W3(x) = Z; Gpp — Py We, (C.6) 
Wı(®) = -PL K(VC(p,) — VC(p})) - Ù; 0: — 070, (C7) 
Ws(x) = -Pe Kiet + gt K (Pe — Py) ha, 


aies ~T 
+P, Kig- + fg K(p, — Po), 


—T_ Bu ee 
U; psy + Ms 4(Ust - Us) 


Ms 


( 
~T_ ms PE 
+U; As- +üs-(U;-— Us) 


—T_ = Oe 
— Uppy, + Aøzr+(Uz - Uz) 


~ A ipa AS = 
+ Uppy + Bz (Uz - Ur: (C.8) 


Due to skew-symmetry of Jp it holds that Wı (x) = 0. The term W2(a) is less than or 
equal to zero whenever the conditions of Lemmas 3.39 and 3.41 are fulfilled. Moreover, with 
~T 


ZpGp = VHy(L) = wg = wg, we get W3(x) = 0. Due to (strict) convexity of Cg i(pg,i), 


the relationship 


1 * * 
= Peni oa Pz,i)(VCg,i(Dg,i) = VC,.(P2.:)) <0 (C.9) 
holds for each i € V,. Thus, the first summand in W4(æ) is less than or equal to zero. The 
same holds for the quadratic terms —(U; — U})'(U; — Uf) and —(Uz — U4)! (Uz - U5), 
which yields W4(a) < 0. 


Finally, since the inequalities RCIP < a g and ju" g* < 0 hold for each convex function 
g = col;{g;} [SDv15, Proposition 3] (see also the proofs of Lemma 3.41 and Theorem 3.44), it 
can be concluded that 


(Heit — Hoi) (Zi — Tiha, S (Wait — Hii) (ti — 27) (C.10) 
and 
(Hei = Hii) (Ti Bi), Si — Hh) (2i — 27). (C.11) 


Accordingly, each row in (C.8) is less than or equal to zero. 


All in all, it follows that V < 0 holds in a neighborhood around æ*, thus x* is stable in the 
sense of Lyapunov [Kha02, Theorem 4.1]. 


D Supplementary Material to Optimal Control 
of Port-Hamiltonian Systems (Chapter 5) 


D.1 Single-Player Case 


H (a) as Natural Control-Lyapunov Function Candidate 


Lemma D.1. Consider the purely dissipative input-state-output PHS 


(D.1a) 


(D.1b) 


with R(x) = R! (x) = 0 and H(x) > 0. Then (D.1) is asymptotically stabilizable at the 
origin by means of a state feedback u(x), if and only if H(a) is a CLF for (D.1). 


Proof. Let g(a) € R” be an arbitrary but fixed vector. Then for all k(x) € R”\{0} it trivially 
holds that k" (a)g(a) = 0 is equivalent to g(a) = 0. Negating this statement yields 


J k(x) ER": k'(x)g(x) > 0 => g'(x)g(x) > 0. (D.2) 


By choosing g(a) = G' (x)V H (x), with G(a) and H(a) as in (D.1), equation (D.2) reads 


Ik(z) ER": k'(a)G'(a)VH(a)>0 => (VA(ax))'G(x2)G" (x)VH (x) > 0. 
(D.3) 


Alternatively, choosing g(x) = YR(z)VH(x) with R(x) = R! (æ) > 0, (D.2) takes the 


form 


Ik(z) ER": k'(e)/R@)VH(2)>0 => (VH(a))"R(x)VH(2) >0. (D4) 
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Now we restrict the vector k(a) in (D.2) to the form k(x) = G'(a)VV(a) and (æ) to 
k(x) = \/R(a)VV (x) with V(x), V(a) > 0, which leads to 


Xy S X2 
3V (x)= 0: (VV(x))'G(x)GT(x)VH(<)>0 = (VH(x))'GG'VH(a) > 0, 
(D.5a) 


nn nn — 
I3V(2) > 0: (VV(a))"R(x)VH (2) >0 — (VH(a))'R(x)VH (a) > 0. 
(D.5b) 


Conjunction of the two statements in (D.5) and contraposition yields X2 A X4 => Xj A Xs, 
which can be written as 


(VH(a))" (Ria) + G(w)G"(2)) VH(z)=0 = GAG. (6) 


The left-hand side of (D.6) holds, if and only if H(a) is no CLF for the PHS (D.1). The 
right-hand side of (D.6) implies that there also exists no V(x) > 0 with (VV (a))'(R(a) + 
G(«)G' (x))VH (x), i.e. there exists no CLF for the PHS (D.1). Thus, we conclude that there 
exists no CLF for (D.1), if H(a) is no CLF for (D.1). The inverse statement is trivial: If there 
exists no CLF for (D.1), then also H (a) is no CLF for (D.1). By Artstein’s theorem [Art83] it is 
known that the existence of a CLF is necessary and sufficient for the asymptotic stabilizability 
of (D.1) at the origin. Hence we conclude that H(a) being a CLF is necessary and sufficient 
for asymptotic stabilizability of (D.1) at the origin 


Openness of W+ 


Proof. To prove that W* is an open set, two auxiliary sets Y* and Yå are introduced which 


can be proven to be open more conveniently. Finally, openness of W* is concluded by a 
canonical projection of Yda: 


Let Y* := {(x,w) € R” x R” : V(x, w) > 0} be the set of all (x, w) with V(x, w) > 0. 
Then Y* is the preimage of Rt, ie. Yt = V-!(R*). Each preimage of a continuous 
function is open whenever the corresponding image is open [Cro05, Theorem 2.9]. Since 
V(x, w) is continuous and R* is open, Y* must also be open. Furthermore, we note that 
Yt is nonempty due to the fact that V(x,0,) = H(x) > 0 is fulfilled by definition for 
all £ € R”\{0}. Now consider the Y* -inner cylinder (see Fig. D.1) with ve (w,W) := 
{(z,w) € Y+ : w €lw, wl, (@,w) € Y+ V  € R"\{0}}. Obviously, Y$ (w, w) < Y+, 
and according to Lemma D.3 (see below), You (w, W) is open for each pair (w, w) € R” x R”. 


D.1 Single-Player Case XLV 


Az 
we 
y+ 
y+ 
ee 
w 


Figure D.1: YT with inner cylinder Ya (w,w) C Figure D.2: Each inner cylinder via (w, W) is an 
Yr. open set. 


With the help of Yå (w, wW), we can define the maximum Y* -inner cylinder 


= U Yh w) (D.7) 


as the union of all possible cylinders Yå (w, W), see Fig. D.1. Since the union of open 
sets is open [Sin19, Theorem 1.1.9], ti is open as well. With the canonical projection 
proj: R” x R” > R”, the set W* can be interpreted as Wt = proj (Ft), i.e. the canonical 


projection of Yi in R”. Since projection maps are open maps [Deo18, p. 5], W* is an open 
set. 


Lemma D.3. The Y*-inner cylinder 


Yr 


cyl 


(w,w) := {(z,w)e Y*: w €lw, wl, (Zw) € Yt Y z e R"\{O}} 


with w, W € R” is an open set. 


+ 


cyl (W, W) = 0. Since empty 


Proof. If w > W, then the interval |w, w| is improper, thus Y 


sets are trivially open, the proof is complete. If w < W, let (x, w’) € eee (w,W) be an 
arbitrary point within the cylinder (see Fig. D.2). Now define 
x” = col; {||x'||}, t= lyons (D.8) 
w” = colj{min{w; — w}, w} — w,}}, eee (D.9) 


where x” denotes the component-wise distances between x; and 0, and w” denotes the 
component-wise distances between w; and the lower or upper bounds w; or Wj, respectively. 
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From Fig. D.2 it can be seen that by definition of x” and w”, it holds that 


(x + ia”, w’) € int Yia (w, W), (D.10) 
(a’ — se”, w’) € int Yia (w, W), (D.11) 
(x, w + Zw") € int YZ (w, wW), (D.12) 
(a’, w — Zw") € int YZ (w, wW). (D.13) 


Moreover, it can be seen that both the “upper” and “lower” subset of You (w, W) are convex 
sets. Thus, it is obvious that we can always construct an open ball B’((x’, w’), £) around 
(a’, w’) with radius 


1 x" 
that lies completely in Yå (w, w). Hence, You (w, W) is an open set. 
D.2 Multi-Player Case 
Proof of Lemma 5.43 
Proof. For linear ISO-PHSs, it holds that 
Xe, = {x € R” : Gl Qa = 0} = ker(G/Q). (D.15) 


Moreover, in [HJ12, Observation 7.1.6] it is stated that for each positive semidefinite matrix 
A € R”*”, the equality x' Ax = 0 holds, if and only if x € ker(A). Thus for each i € P, 
condition (5.108) is equivalent to 


N 
ker(@7Q) Nker(QTRQ) NO | (N ker(Y; -Q'G;S;'G}Q) | = 0. (D.16) 


j=1 
ji 


D.2 Multi-Player Case XLVII 


Due to the fact that T; > 0 and ker(G] Q) C ker(Q'G;S7'G]Q), (D.16) can be written 


as follows: 


N N 
U | ker(G@7Q) N ker(Q'RQ)N | | ker(Q'G;S7'G]Q) 
z ji 
N N 
=ker(Q'RQ)N |] | ker(G@7Q)/ | A ker(Q'G;S;'G; Q) 
i=l j=l 
jFi 


N N N 
=ker(Q’RQ)/N (N wi @1Q)) n | LJ | N ker(Q™G;S;'G7Q) 
| OE 
=0. (D.17) 


With ker(QA) = ker A for each regular matrix Q together with Lemma D.4 (see below), 
(D.17) is equivalent to (5.117). 


Lemma D.4. For each positive definite matrix C € IRN, it holds that 


ker(AC) Nker(BC) = 0 <> ker(A)Nker(B) = 0. 


Proof. The left-hand side of (D.18) can be written as follows 


Ax € R”\{0}: (ACz = 0)A(BCa = 0). (D.19) 


With z := Cx this is equivalent to 


Az € R”\{0}: (Az = 0) ^ (Bz = 0). (D.20) 


Accordingly, (D.20) states that the kernels of A and B are disjoint. This is equivalent to the 
right-hand side of (D.18) . 


Proof of Corollary 5.45 


Proof. As a consequence of (5.94), it follows that V (a) is equivalent to the vector of value 
functions, if and only if Y;(t) = 1 for all i € P. With (5.105), this means that 


N 
q? (æ) +4; (@) +X Ele) + a? (a) = 0, (D.21) 
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which is equivalent to 


1 avi)" at Vi _ yn (OU\" RE AC)) 
-3(&) Hase) "ce el =) ls 
jżi 
ƏV; \ T öH 1 
+ (Z) Ue) -RoE + 54) 
=0. (D.22) 
By inserting (5.118), (5.126), and multiplying by —1, we get 
öH\' +05; OH ET 
(&) =. +) w) 
N T T 
OH 1ôE \ {dH , (88;\" 
Fi 
dz; oH aH\! dH 1 
T a Pe ans 
Dw! E) - RE) E +2 (Fz) Rea) Gr - Fhe) 
=0. (D.23) 


Rearranged by w,, this yields 
OB, 0B; \" 
al a % uv i 
(w7 dr ) K;(«) (( Jz ) w) 
öH\' az; N" OH\'  ~08;\ 7, .f08:\" 
+2( (32) K(x) (5) w) (3) twa Kyte) (5) wi 


=0, (D.24) 


thus (D.24) can be written in the form (5.121)-(5.126). 


E Supplementary Material to Case Studies 
(Chapter 6) 


E.1 Benchmark Model Parameters 


Table E.1: Numerical values of the nodal parameters used in Case Studies I and I. 


No. Type A; Ti Xai-Xai Tao, 


1 L 1.61 = = = 
2 S 1.68 24.73 7.28 0.127 
3 S 1.61 20.84 7.41 0.185 
4 T 1.65 4.48 = = 
5 L 1.60 = = = 
6 S 1.54 22.10 6.84 0.147 
7 L 1.36 = = u 
8 S 1.40 23.62 7.63 0.173 
9 S 1.38 21.17 7.44 0.168 
10 S 1.61 24.61 7.14 0.150 
11 T 1.44 4.91 = — 
12 L 1.44 = _ _ 
13 L 1.65 = = = 
14 T 1.50 5.47 = = 
15 I 1.62 5.37 = = 
16 tT 1.50 4.55 = = 
17 I 1.33 5.06 = = 
18 I 1.63 4.49 _ = 
19 S 1.33 26.97 6.95 0.152 
20 L 1.33 = = = 
21 S 1.29 25.00 6.58 0.150 
22 I 1.45 4.56 = = 
23 L 1.28 = = = 
24 I 1.39 4.55 = z 
25 T 1.31 5.06 = = 
26 T 1.33 4.66 = = 
27 L 1.24 = — = 
28 L 1.68 = = = 
29 S 1.35 22.51 6.72 0.186 
30 S 1.66 22.54 6.46 0.150 
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Table E.1: Numerical values of the nodal parameters used in Case Studies I and II (cont’d) 


No. Type A; Ry Xai— Xa: Tao: 


31 L 1.32 = = = 
32 S 1.50 25.82 6.64 0.180 
33 I 1.69 4.07 = = 
34 S 1.69 25.43 7.04 0.189 
35 L 1.43 = = = 
36 T 1.60 4.44 _ _ 
37 S 1.67 25.13 7.63 0.149 
38 L 1.46 = = = 
39 S 1.69 26.98 7.08 0.147 
40 S 1.29 25.48 7.58 0.144 
41 S 1.39 25.31 6.40 0.147 
42 tT 165 4.94 = = 
43 L 1.51 — = = 
44 S 1.33 23.21 6.65 0.162 
45 aD 1.30 4.07 = = 
46 I 1.36 4.14 _ — 
47 L 1.48 — = T 
48 S 1.28 22.43 7.28 0.189 
49 T 1.61 4.54 = = 
50 T 1.37 4.53 = = 
51 L 1.35 = = = 
52 L 1.41 — = = 
53 T 1.58 5.37 = = 
54 L 1.59 = = F 
55 S 1.40 20.56 7.07 0.123 
56 L 1.33 = = a 
57 L 1.25 = = = 
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